Add appellate support to recap and juriscraper — Funded

[freelawbot]

Issue by PlainSite
Saturday Jul 19, 2014 at 16:05 GMT
Originally opened as https://github.com/freelawproject/recap-extension-2.0/issues/10

---

Right now Firefox has appellate parsing features and Chrome doesn't. It would be good to get them in the same state.

[freelawbot]

Comment by mlissner
Saturday Jul 19, 2014 at 18:16 GMT

---

+1, marking for 2.0 release.

[mlissner]

This old ticket needs resolution. In a few weeks neither Chrome nor Firefox will support appellate. That's just the way it has to be for a bit, but the good news is it puts more pressure on getting this issue (from 2014) resolved.

[johnhawkinson]

@FiloSottile have you any time to take a look at this, since you wrote the appellate support for the now-deprecated Firefox extension?

[sjschultze]

Beginning work in https://github.com/freelawproject/recap-chrome/tree/appellate-dev

[sjschultze]

Some todos:

• add isDocketQueryUrl support and corresponding logic:
  freelawproject/recap-chrome@75bbdf2#diff-4e91f732f3d6c295fa99b1b972e0f53aL72

• Same for the doc1/docs1 pages:
  freelawproject/recap-chrome@75bbdf2#diff-4e91f732f3d6c295fa99b1b972e0f53aL56

[sjschultze]

Basic upload and parsing of CAVC dockets is now working in the appellate-dev branch, so we can do development without incurring PACER fees. Example case: https://www.courtlistener.com/docket/7625727/michael-p-long-v-robert-l-wilkie/

For reference:

• freelawproject/recap-chrome@dc08c41
• freelawproject/juriscraper@9df558f
• freelawproject/juriscraper@14923ed

Next step: get PDF uploading working

[sjschultze]

Unfortunately, I'm not going to have much time to work on this issue. I wanted to note one special thing about CAVC: it never shows receipt pages. I'm not sure what rely on receipt pages for, but we do check for them and warn if they are not enabled:
https://github.com/freelawproject/recap-chrome/search?q=receipt&unscoped_q=receipt

See also this maybe-related issue in the courtlistener code for server-side:
freelawproject/courtlistener#867

(oh and I tried forcing them on with javascript:document.cookie="PacerPref=\"receipt=Y\"" but it made no difference)

[mlissner]

Thanks for pushing the ball forward, Steve. CAVC is an interesting edge case. We relied on receipt pages for at least the document and attachment numbers, so that might indeed explain #867.

[mlissner]

Small thing, but when this launches, we need to:

• Do a notification of some kind in the browser notifying people that appellate support is here.

This is important because some people turn RECAP on and off as they use the courts to avoid uploading things they don't want to, and we don't want them accidentally learning that appellate is now uploading things (after they've uploaded something they didn't intend to).

[mlissner]

I forgot this, but the Court of Appeals for Veterans Claims runs a free site:

https://efiling.uscourts.cavc.gov/cmecf/servlet/TransportRoom?servlet=CaseSearch.jsp

I don't know how different it is (if at all) from the regular appellate sites, but it might be a way to save on PACER fees.

[jgrisham]

• Do a notification of some kind in the browser notifying people that appellate support is here.

Would something similar be acceptable now, to show new/intermittent users that appellate support does not yet exist?

Tangentially, see my comments about documentation in the post-closeout comments on a duplicate of this issue: #299 (comment)

---

I'm happy to try and help (and not just to disguise my embarrassment after spending 4 hours trying to get RECAP to upload 5th circuit documents / dockets over the past two days).

[mlissner]

Jim, are you a developer enough that you could evaluate what we've got? We have a version that's maybe 80% of the way there, but we haven't had the developer time to take it over the finish line.

I thought we had a warning on the icon when you were at an appellate court, but I haven't checked in awhile. I didn't think we removed that though.

[jgrisham]

@mlissner: Evaluate, yes. No promises on fixes, though. ;)

I'm hardly an expert with modern tools (especially regarding GitHub), but I'd be happy to dig in if someone were to point me towards any existing documentation and code branches (and upload or forward a copy of any potentially relevant notes / e-mails).

(I'd hate waste time re-inventing the wheel regarding any architectural / API research that was attempted in the past.)

[jgrisham]

Follow-up… even if the code is only 80% complete now… perhaps a portion can be quickly activated in order to hoover up any documents that are purchased in the near-term; later we add the backend capability to automatically link those document numbers to specific docket and case information.

Alternatively, if the code for recording docket data is closer to completion than the document code, we can at least get some appellate dockets added to the database. Publicly-available dockets would still be a great public service before the actual contents of documents are available. In the meantime, if the dockets revealed document numbers, users without PACER access may be able to use that data to find selected documents hosted elsewhere on the Internet.

[mlissner]

@litewarp did some work on appellate that may be worth looking at, or perhaps not. He may chime in with details about how far along the work is. His fork is here:

https://github.com/litewarp/recap-chrome

The problem with this fork is that it has a LOT of other stuff in it, so there was no way to carefully review it before launching it to our 20k users.

I honestly think a fresh implementation of appellate is easier than trying to tease out the appellate work in that fork (it's 400 commits behind main), but I know @litewarp put a lot of work into it, so I want to at least make sure that somebody looks at this and thinks about it while we're working on appellate.

[litewarp]

@mlissner I know I've voiced this before, but the existing recap code is so far behind modern JS that there is not going to be any simple migration path, nor a comprehensible way to add new features to recap.

A lot of the "other" stuff in that fork (from 2+ years ago!) was an effort to move past jquery and aughts-level-monkey-patched-callbacks without changing the functionality. It's why I kept and ensured compliance with all the tests -- new formula; same great taste.

Also, I wrote it in typescript and used a bundler. Migrations like this should bolster your confidence in shipping your app. Type checking produces enormous benefits as well as taking advantage of all the fanciness and improvements in modern day JavaScript.

Idk, bugs happen on new versions. But at this point even the "modernized" code I wrote would be somewhat stale.

I would love to continue to work on recap. But it's so frustrating to work on that I understand why you nor another developer have touched it. But that's literally the reason I wrote that fork!

tl:dr -- I don't know how you move recap forward now without addressing that "other" stuff. Maybe time to rip of the band aid?

[mlissner]

It's not the older-style JS that was preventing work on these issues, it was the lack of time, and if I'm honest, I find dependency management, bundling, and the rest of it to be kind of a headache that prevents me from working on things, because I'm always half expecting NPM to be the wrong version or some dependency to have a vulnerability or who knows what. I'm leaning towards ripping react out of CourtListener for this reason, for example, and to just use HTMX instead — it's SO MUCH simpler.

But typescript is probably useful to some degree, and modern JS without the callback hell is certainly nice. Won't deny that, and I think we're slowly moving towards that.

The other thing that's coming up is manifest v3, which will require a pretty big redo some time early next year. I think when we get to that we'll be wise to take another look at your branch and see how much of it we can re-use, but we've gotta do it bit by bit, however that's possible. Huge rewrites without planning are really just too scary.

[mlissner]

We're making some progress on this over in #296.

But we've got some fresh bugs in #322, #323, #324, and #325. Appellate is hard.

[litewarp]

@mlissner I will for-the-last-time chime in to say that the work you are doing now will need to be rebuilt again (and within the next six months or so!) when you upgrade to the v3 manifest. I think your hatred of npm is both warranted and a poor excuse to not modernize the recap extension.

In v3 you're going to have to bundle external dependencies, use fetch instead of XMLHttpRequest, and migrate the code to using service workers and ES modules.

It just seems short-sighted to me to do appellate recap on a separate track as the v3 migration, especially when most of my changes make v3 adoption easier. Just trying to save you time, but 🤷

[mlissner]

I hear ya. We'll get moved over eventually, and I appreciate you pushing despite me. I'm not in a rush to get to v3 because Firefox is still working on supporting it.

For the modernization stuff, I'm really not sure it makes a huge difference whether we do it now or later. The code we're adding right now for appellate support is all very similar to what we have for district courts, plus a bunch of helper functions that'll move across smoothly to v3 when we get to that. I don't think we're digging ourselves in deeper, in other words.

[litewarp]

You've been at this longer than me, so I trust you'll get it done.

To my ears though, your last paragraph sounds a lot like "I'm pretty sure we're not making the boat sink any faster by adding a second floor dining room, so ...."

Gl and let me know if you need help or have questions!

[mlissner]

Ha. I think it's more like we're not making it sink faster by adding a leaf to the dining table, but we'll see. It'll get there!

[mlissner]

This is done and launched. Closing and walking away from my desk for a while.