[YOUTRACK] Add YouTrack Issues

[CanisHelix]

Add support for YouTrack Cards/Issues as per #9997

A new Service Token and test is added, InvalidResponse of processing added when a -1 is returned, and cached for 10 seconds. This is expected when YouTrack isn't finished and is documented at https://www.jetbrains.com/help/youtrack/devportal/resource-api-issuesGetter-count.html#create-IssueCountResponse-method

If the response returns "count": -1, it means that YouTrack hasn't finished counting the issues yet. Wait for a bit and repeat the request.

For CI purposes shields.youtrack.cloud was created and is used throughout all tests as this instance is under our control. The guest user is enabled read-only so that the CI tests can work in Github Actions without needing a token, but prevents people breaking the tests.

Only concern is whether or not cachedSeconds is working as IntelliJ IDE reports this doesn't match InvalidResponses definitions, but seems to work. And also, during testing when the cloud instance is idle for long periods, a timeout is received on the first test of a day, as such I added 'timeout' as a possible test response.

[github-actions]

	Warnings
⚠️	This PR modified the server but none of its tests. That's okay so long as it's refactoring existing code.

	Messages
📖	✨ Thanks for your contribution to Shields, @CanisHelix!
📖	Thanks for contributing to our documentation. We ❤️ our documentarians!

Generated by 🚫 dangerJS against 8bcc3da

[chris48s]

So I checked this out locally.

Before I give a round of comments on the diff, I tried requesting
http://localhost:8080/youtrack/issues/DEMO?youtrack_url=https%3A%2F%2Fshields.youtrack.cloud&query=bug%20%23Unresolved
a bunch of times. For me, this just says processing every time - just a -1 back from the API every time.

Do you get the same?

In order for this to be a viable badge, I think we can work round a bit of flakyness, but it is going to mostly need to return a valid badge.

I don't want to spend too much time bikeshedding the code if this just isn't going to work reliably.

[CanisHelix]

It seems the YouTrack API is a little flaky there. The word 'bug' doesn't exist in any of the cards so instead of returning 0 it returns -1 constantly.

If I use Type: Bug #Unresolved instead of Bug #Unresolved to search for bug type cards it works.

http://localhost:8080/youtrack/issues/DEMO?youtrack_url=https%3A%2F%2Fshields.youtrack.cloud&query=Type%3A%20Bug%20%23Unresolved

Returns 0 for me.

Searching for manage #Unresolved which translates to URL http://localhost:8080/youtrack/issues/DEMO?youtrack_url=https%3A%2F%2Fshields.youtrack.cloud&query=manage%20%23Unresolved returns 7 for me.

I'll update the example as that was bad and an oversight. One thing I have noticed is that in their own UI tags like #Unresolved work very well but via the API seems flaky. state: Unresolved is much more stable on the API from this mornings testing.

[chris48s]

It is a bit annoying that -1 can mean both "processing" and also indicate some kind of invalid query.

One other thing I have noticed is that once the query has been evaluated, it only seems to stay cached upstream for quite a short amount of time. Maybe a minute or so. Then you're back to "processing".

I know there is some sunk cost here because you've already put some time into implementing this. Do you think what's here right now is reliable enough you'd put it in your README? It feels to me like this is a bit too flaky to be useful/supportable at the moment.

Another thing I have noticed is: if you get back a -1, you seem to be able to make a second request and get back a valid result pretty quickly. Do you think that is true more widely, or does this only happen on the projects I'm looking at because they have a really small search space?

[CanisHelix]

If I did not sink time into this, we would not know about how flaky their API is. It's been a good learning curve of some areas of Shields new to me too.

YouTrack doesn't have a real DB for its backend which may be why they do not cache very well. In the current state of their API responses, I would agree that this is not good enough for a README.md.

In general they advice retrying after a few seconds with a -1, which I believe is okay for an application that would use their API. But badges needs to be quick. I did consider one last idea which was the following logic:

1. Call API
2. If not -1, render. If -1, step 3
3. Wait x seconds and retry
4. Loop a maximum x times before giving up as invalid.

Given that -1 can be returned instead of 0 due to just a bad query (syntax wise valid, but data wise invalid), I was hesitate to implement this logic to avoid an infinite loop scenario. With the x retries attempts, I am concerned about the negative performance it would have on shields.io. If a retry mechanic is ok, I can give that a shot. If not, we can close this out due to upstream not being reliable enough at this time.

[chris48s]

Before we go down that route, do you have an example of a project/query which involves searching a large number of issues to evaluate the query?

I'd like to get an idea of how long/short an amount of time we would need to wait before retrying in that case before we put more time into coding solutions.

[CanisHelix]

After testing and discussion on discord this PR will go into Draft Mode until such a time that YouTrack's API is more reliable. In the current state an API can be processing anywhere from 1 to 20 seconds, this is not an acceptable response time from the upstream API. I will leave my branch up and periodically update it. Hopefully JetBrains improve their API in the future.