Allow custom decks via JSON [Feature request]

[LukeMoll]

Since CardCast has shut down and (as of time of writing) api.cardcastgame.com returns NXDOMAIN, there is currently no easy way to add custom decks other than importing them into the database. To avoid reliance on third parties in the future, adding custom decks via URL in JSON format could be a way to achieve this:

/addcardcast <deck code> is replaced by /addurl <url>, where <url> is a link to a hosted JSON file (or an API that returns JSON for that deck). The format of the JSON could be similar to Cardcast's to aid compatibility, although finding documentation on that is harder now.

Obviously for some servers the ability to load these files is a concern, so I propose an option (perhaps in build.properties) firstly to allow/disallow this feature, and later on to whitelist certain domains.

[devgianlu]

That is a fantastic idea, I can write a PR if @ajanata will merge it.

The shutdown of Cardcast removed a key feature in my app and having a way to create custom decks directly on the phone would be awesome.

[ajanata]

I like the idea and I'll merge it (probably not difficult to change the existing cardcast code, which I won't delete when I disable the commands later today), but I will not enable it on my own servers. I've had too many issues with DDoS in the past, which stopped when I put the servers behind Cloudflare... Letting users make the servers connect to arbitrary URLs will expose their actual IP addresses, which would make DDoSes possible again.

There also should be some sort of check on the content-type and content-length response headers before the entire response is downloaded, to avoid intentional large responses trying to crash the servers.

Now, if there'd be a whitelist of domains (in addition to the size checks), I'd consider having that enabled; e.g. gist.github.com is trustworthy.

Obviously this wouldn't be as easy to use for the end user, but it'd at least enable the functionality again.

I still have some sample cardcast json files somewhere. The existing code does hit two endpoints (to get metadata about the deck (mainly its name), and to get the cards themselves). This should be combined into a single file for something like this. Additionally, deck ID numbers will need to switch to something like the card ID numbers (monotonically increasing), but also probably still be negative. And the watermark needs some thought. It's definitely doable without a huge amount of effort.

I believe the format for the cardcast files were something like (at least for the relevant fields)
(info endpoint)

{
  "name": "name of the deck",
  "description": "I actually don't see this in my example files, so... remove references to it?"
}

(cards endpoint)

{
  "calls": [
    {
      "text": [
        "",
        " and ",
        " are the next hit comedy duo."
      ]
    },
    {
      "text": [
        "What's that smell?",
        ""
      ]
    }
  ],
  "responses": [
    {
      "text": [
        "The biggest, blackest dick."
      ]
    },
    {
      "text": [
        "Two midgets shitting in a box."
      ]
    }
  ]
}

The calls (black cards) are joined with ____, and len(text) - 1 is the pick value (there must always be at least two things in that array). The responses (white cards) are simply taken as-is.

[devgianlu]

What about providing the JSON directly to the server instead of fetching it from an URL?

We could have both: either from an URL if the user uploads it to a trusted source or from the user itself. Would that expose any vulnerability?

[ajanata]

Direct uploads should be fine. In fact, that should provide size protection automatically if the servers are being reverse proxied (mine are behind nginx).

Honestly it might be worth just overhauling the system entirely to take black cards in ____ and ____ are the next hit comedy duo. format directly (and it can count the number of ____ to determine the pick amount), and just a list of white cards. Possibly in two text fields.

Shit, I almost want to do this myself (haven't had motivation to do much of anything in months), but I have too many other things going on this weekend. :(

[devgianlu]

I've done some work with #241.

[ayca-kaygusuz]

If I am reading the code right, we can't currently upload custom decks directly on the website, can we? As in, @devgianlu's code hasn't been merged with master yet?

[devgianlu]

@StarburstGalexy When #241 will be merged, you'll be able to upload custom decks to the server and use them in game.

[ajanata]

I'm taking a look at it this weekend. The last week was incredibly busy at my day job...

[devgianlu]

@ajanata Do you mind if I reformat the files I'm modifying to use 2 spaces indentation consistently? Various PRs messed the formatting quite a lot. What's the minimum Java version this needs to run on? There are some optimizations that can be done in this regard.

[ajanata]

Do you mind if I reformat the files I'm modifying to use 2 spaces indentation consistently? Various PRs messed the formatting quite a lot.

Aw crap, I wasn't checking when I was merging stuff... Go for it.

What's the minimum Java version this needs to run on?

My servers are still on 8. I'd prefer it if it'd still work there, cuz I haven't tried upgrading any of them and fully expect things to break.

[devgianlu]

I've setup my own server based on #241. Available at: https://pyx.gianlu.xyz/ZY/, the Android app has also been updated for beta testers (3.2.0).

[BrozzSama]

@devgianlu I don't know what the future plans are for #241 if it still going to be developed might good to implement the solution of having the system identify cards with "___" in order to have compatibility with CRCAST JSON format, as @ajanata already suggested.
For the time being it could be useful to provide a helper script in your branch like the one I posted below that converts CRCAST decks into PYX compatible json, since it seems like there is no "official" way to do this as of right now.

import json
from re import split

in_file = 'input.json'
out_file = 'output.json'

with open(in_file) as json_in:
    data = json.load(json_in)

data['watermark'] = data['deckcode']
data['responses'] = data['whites']
data['calls'] = data['blacks']
del data['deckcode']
del data['blacks']
del data['whites']

for response in data['responses']:
    response['text'] = [response['text']]

for call in data['calls']:
    if '_' in call['text']:
        call['text'] = split(r'_+', call['text'])
    # No fill? No problem.
    else:
        call['text'] = [ call['text'], '' ]

with open(out_file, 'w') as json_out:
    json.dump(data, json_out)