Building JAX port of Detectron2

[INF800]

Hi, I have been using Detectron2 for almost a year for training large scale models and their results (mask rcnn) always fascinate me. Like FastAI it is one of the most flexible frameworks I have used for deeplearning.

I wanted to understand internals of both JAX and Detectron2 for some time and I think building flax port for Detectron2 will get me there.

Can I know if I can get any support from the team for solving issues if I get stuck.

Timeline:

6mos to 1year starting from Feb 2022. I have couple of commitments that hinder me from full time contribution but can spare weekends and some weekdays for the project. The idea is to slowly but effectively build the project.

Motivation:

Building mature frameworks for training models (stress on large-scale) is always critical for both research and industry. And JAX is currently the hottest topic I want to learn by doing.

[INF800]

Linking discussion from Detectron2 facebookresearch/detectron2#3817

[avital]

Hi @INF800 -- we'd love to see something like Detectron2 ported to JAX/Flax. (Though I personally don't know almost anything about that project).

What kind of support would you be looking for? Then we can ask around and see if someone can help in the best way.

[INF800]

Hi thank you for the response. I will NOT be needing any help for understanding the already implemented code in Detectron2 (or any python code as such). But Instead, I will be needing help in porting pytorch code to JAX (i.e FLAX) with best practices. In JAX you can consider me at beginner level and for pytorch I am at intermediate level.

…

On Thu, 23 Dec, 2021, 7:42 pm Avital Oliver, ***@***.***> wrote: Hi @INF800 <https://github.com/INF800> -- we'd love to see something like Detectron2 ported to JAX/Flax. (Though I personally don't know almost anything about that project). What kind of support would you be looking for? Then we can ask around and see if someone can help in the best way. — Reply to this email directly, view it on GitHub <#1740 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AK4GSXIOWSHDB67YW2FDA4TUSMU35ANCNFSM5KTS46UQ> . Triage notifications on the go with GitHub Mobile for iOS <https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675> or Android <https://play.google.com/store/apps/details?id=com.github.android&referrer=utm_campaign%3Dnotification-email%26utm_medium%3Demail%26utm_source%3Dgithub>. You are receiving this because you were mentioned.Message ID: ***@***.***>

[avital]

Folks on our GitHub discussion board are pretty helpful, and both folks that work on JAX and Flax, as well as knowledgable community members can help. The upside this way is that other people can benefit from the learning. I propose we start with that, and if you start making progress and would benefit from more intensive support we can see what we can do. How do you feel about that?

[INF800]

Folks on our GitHub discussion board are pretty helpful, and both folks that work on JAX and Flax, as well as knowledgable community members can help. The upside this way is that other people can benefit from the learning. I propose we start with that, and if you start making progress and would benefit from more intensive support we can see what we can do. How do you feel about that?

Perfect! I will open up related issues/discussions and link them to this issue to keep track of them. As said before I will be starting it actively from Feb 2022. Until then will try to understand more about JAX and Detectron2 and you can expect me to open up multiple discussion topics related to JAX :)

[avital]

Great! One possible first step is only to port one particular model in a standalone codebase, such as Mask R-CNN, and see what we need to do from there.

[INF800]

Great! One possible first step is only to port one particular model in a standalone codebase, such as Mask R-CNN, and see what we need to do from there.

Makes sense I was planning to do the same. Will update here once I create it.

[saberkun]

https://github.com/google-research/scenic is a flax computer vision code base as a reference.

[INF800]

https://github.com/google-research/scenic is a flax computer vision code base as a reference.

Thanks. Detectron2 internally uses fvcore https://github.com/facebookresearch/fvcore. Is this project similar to fvcore?

[marcvanzee]

@INF800 I moved your issue to a "Show and Tell" Discussion Thread, which seemed more appropriate. Looking forward to hearing about your progress!