Skip to content

How to get hypothesmith to generate a FunctionDef with arguments? #31

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
julian-goettingen opened this issue May 31, 2023 · 2 comments

Comments

@julian-goettingen
Copy link

julian-goettingen commented May 31, 2023

I have been experimenting with this library for a bit for purposes of testing refactoring tools. I can generate lots of extremely weird identifiers for all kinds of things, but the structure of everything generated seems fairly simple - too simple.

For example, I am trying to generate function signatures. I imagine many possibilities with the arguments - they can have annotations, positional-only-args, the **kwargs-keyword, default-arguments, etc etc.

Here is my code to generate function-definitions that have at least something in their argument brackets (execute with pytests -s flag to show the prints):

import hypothesmith
from hypothesis import given, settings, HealthCheck, assume
import libcst
import re

s = hypothesmith.from_node(node=libcst.FunctionDef, auto_target=True)

@given(code = s)
@settings(suppress_health_check=[HealthCheck.filter_too_much, HealthCheck.too_slow], max_examples=5000)
def test(code):
    assume(re.search(r"\([^)(]+\)\s*:", code) is not None) # should match 'def foo(x):pass' but not 'def foo():pass'
    print("------")
    print(code)
    print("------")

This calculates for a long time but finds absolutely nothing. What am I doing wrong? Or did I misunderstand the purpose of this library?

@Zac-HD
Copy link
Owner

Zac-HD commented May 31, 2023

Hi Julian! This is definitely something that I'd like to work, but as the readme says

This is definitely pre-alpha, but if you want to play with it feel free! You can even keep the shiny pieces when - not if - it breaks.

so I think the answer is just that we're currently defaulting to a pretty poor/narrow automatically-inferred strategy for FunctionDef nodes. The obvious solution is just to add a hand-written strategy to https://github.com/Zac-HD/hypothesmith/blob/master/src/hypothesmith/cst.py - want to open a PR?

Unfortunately hypothesmith is currently a tier-three oss project for me - priority is something like {work, phd}, {hypothesis}, {hypofuzz, shed, pytest, flake8-trio}, {hypothesmith, hypothesis-jsonschema, ......}, so I'm unlikely to write it myself any time soon; very happy to review and help with PRs though!

@julian-goettingen
Copy link
Author

Thank you for the quick response and the pointer on how to go about implementing it.
I'll put it in my own personal backlog, though I don't suppose that PR is happening this week.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants