Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Dependency check problems in v6.16.3 and v6.16.4 #339

Open
alekdavis opened this issue Nov 4, 2024 · 2 comments
Open

Dependency check problems in v6.16.3 and v6.16.4 #339

alekdavis opened this issue Nov 4, 2024 · 2 comments

Comments

@alekdavis
Copy link

alekdavis commented Nov 4, 2024

The following code illustrates nested dependency. Test A completes request, but fails assertion. Test B depends on some condition of test A and test C depends on some condition of test B:

### Test A
# @name a
POST https://httpbin.org/anything

?? status == 201

### Test B
# @name b
# @ref a
{{
  exports.$cancel = (aResponse?.statusCode !== 201);
}}
PATCH https://httpbin.org/anything

?? status == 200

### Test C
# @name c
# @ref b
{{
  exports.$cancel = (bResponse?.statusCode !== 200);
}}
GET https://httpbin.org/anything?b={{bResponse?.statusCode}}

?? status == 200

In , v6.16.2, the result is what you'd expect: both tests B and C get skipped:

---------------------

=== Test A ===

POST https://httpbin.org/anything
=> 200 (2224 ms, 590 B)
✖ status == 201 (AssertionError [ERR_ASSERTION]: status (200) == 201)

---------------------

=== Test B ===

PATCH https://httpbin.org/anything
○ Test skipped

---------------------

=== Test C ===

GET https://httpbin.org/anything?b={{bResponse?.statusCode}}
○ Test skipped

In In , v6.16.3 and 6.16.4, test B gets skipped, but the pre-request script of test C does not get processed due to the ReferenceError: bResponse is not defined error. So, the code to set the $cancel variable does not complete and test C is executed:

---------------------

=== Test A ===

POST https://httpbin.org/anything
=> 200 (1528 ms, 590 B)
✖ status == 201 (AssertionError [ERR_ASSERTION]: status (200) == 201)

---------------------

=== Test B ===

PATCH https://httpbin.org/anything
○ Test skipped

---------------------

=== Test C ===

GET https://httpbin.org/anything?b={{bResponse?.statusCode}}
✖ bResponse is not defined (ReferenceError: bResponse is not defined - Object.userJS (c:\PROJECTS\Test\HttpYacTest\test.http:21:22)

However, test C shows everywhere as successfully executed:

image

I think the test dependency logic in v6.16.2 was mostly correct. The only issue I seem to see with v6.16.2 is that it would try to invoke a failed test multiple times during a test run (for every @ref). But v6.16.3 and v6.16.4 do not work with my logic at all unless I'm doing something wrong.

@alekdavis
Copy link
Author

The following script could be helpful for testing:

### A
# @name a
POST https://httpbin.org/anything

?? status == 201

### B
# @name b
# @ref a
{{
  exports.$cancel = (aResponse?.statusCode !== 201);
}}
PATCH https://httpbin.org/anything

?? status == 200

### C
# @name c
# @ref b
{{
  exports.$cancel = (bResponse?.statusCode !== 200);
}}
GET https://httpbin.org/anything?b={{bResponse?.statusCode}}

?? status == 200

### D
# @name d
# @ref b
{{
  exports.$cancel = (bResponse?.statusCode !== 200);
}}
GET https://httpbin.org/json?b={{bResponse?.statusCode}}

?? status == 200

It should produce test results:

---------------------

=== A ===

POST https://httpbin.org/anything
=> 200 (545 ms, 590 B)
✖ status == 201 (AssertionError [ERR_ASSERTION]: status (200) == 201)

---------------------

=== B ===

PATCH https://httpbin.org/anything
○ Test skipped

---------------------

=== C ===

GET https://httpbin.org/anything?b={{bResponse?.statusCode}}
○ Test skipped

---------------------

=== D ===

GET https://httpbin.org/json?b={{bResponse?.statusCode}}
○ Test skipped

@alekdavisintel
Copy link

alekdavisintel commented Dec 5, 2024

@AnWeber I think I figured out a workaround. The problem now is apparently that when a test gets skipped neither the test nor the testResponse variable gets defined, so I added a check to see if the variable is undefined before testing its properties:

### A
# @name a
POST https://httpbin.org/anything?x=a

?? status == 201

### B
# @name b
# @ref a
{{
  exports.$cancel = (aResponse?.statusCode !== 201);
}}
PATCH https://httpbin.org/anything?x=b

?? status == 200

### C
# @name c
# @ref b
{{
  exports.$cancel = (typeof bResponse === 'undefined' || bResponse?.statusCode !== 200);
}}
GET https://httpbin.org/anything?x=c

?? status == 200

### D
# @name d
# @ref b
{{
  exports.$cancel = (typeof bResponse === 'undefined' || bResponse?.statusCode === 200);
}}
GET https://httpbin.org/json?x=d

?? status == 200

Test results:

---------------------

=== A ===

POST https://httpbin.org/anything?x=a
=> 200 (437 ms, 610 B)
✖ status == 201 (AssertionError [ERR_ASSERTION]: status (200) == 201)

---------------------

=== B ===

PATCH https://httpbin.org/anything?x=b
○ Test skipped

---------------------

=== C ===

GET https://httpbin.org/anything?x=c
○ Test skipped

---------------------

=== D ===

GET https://httpbin.org/json?x=d
○ Test skipped

It seems to have done the trick, but I am still wondering if this is a bug.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants