Make Panoptes ArcFinder Demo

[drphilmarshall]

Things we need to do or get:

• Example test and sim gri PNG images for ArcFinder search ( @anupreeta27 )
• Manifest with feedback messages
• Classification interface with feedback messages
• Make a few people "gold standard" classifiers so we can test the creation of the dud set!

Also:

• Talk to Marten about Nero (online SWAP) status
• Ask Grant about launch/episode strategies
• Find out latest on project timelines from Chris L

Then:

• Identify which features are still missing

[aprajita]

Ignore the order of the checklist, and the last thing will not be a "then" but done as I build.

it's missing a build second demo site which i am ~ 60% through.

[aprajita]

First issue on updates solved by Cam
zooniverse/panoptes#1477
(should submit to https://github.com/zooniverse/Panoptes-Front-End for similar issues)

SW desirable/essential things for the interface/FrontEnd

• tutorial:
  • no current means of including this (essential) AV to follow-up possibilities for this
• speed:
  • need to remove confirmation screen (essential) AV to issue Is it possible to disable the "Thanks! Your classification" page/step zooniverse/Panoptes-Front-End#1902
  • image load (desirable, needs actual sample data to do speed checks before any query started)
• queue up next image:
  • was nice in SW1&2 but not really necessary
• pan & zoom
  • was useful particularly for blind search (desirable) check with sample blind & targeted data
• testing feedback
  • more to add when proper example & gold standard data ingested (essential)
• displaying colour settings & loss of quick dashbaord
  • investigate how different colours will be shown (different versions accommodated in image upload)
  • investigate how subtracted/unsubtracted images will be shown (different versions accommodated in image upload)
  • investigate possibilities of postage stamps being shown other than play button
• changing image classification threshold to NERO input
  • AV liaise with Marten
• back button
  • there is a back button but it's greyed out, AV to ask how to put this on/off or remove it

Other things

• data ingestion outside Panoptes Project Builder (currently limited to 10,000 files)
  • AV to investigate best means of uploading data

@drphilmarshall @anupreeta27
I've added you as collaborators as I build in the non-demo site, note the only variables in the display are the

• background images
• avatar/icon

i used the images from the landing page of the site and the facebook avatar

we can upload what we wish here so have some thoughts as we all weren't keen on the avatar
we have no control over fonts etc, so we should think if we want to make it look more like the old SW interface (or at least the aspects we liked, e.g. font etc. and then i'll find out how/if we can modify these)

OLD demo site: http://demo.zooniverse.org/panoptes-front-end/#/projects/Aprajita/SpaceWarpsRings/classify?workflow=1115
NEW demo site: https://www.zooniverse.org/projects/aprajita/swdemo/classify/

[aprajita]

@anupreeta27 @drphilmarshall I have uploaded Anu's test data

• no obvious way to upload feedback info
• no obvious way to implement feedback in project builder
  • raise issue AV
• no obvious way t provide different types of training subjects in 'gold mode'
  • discuss with Marten AV
• images need to resized
  • issue pan & zoom features query AV
  • resize pngs AM
• settings may need tweaking (data will be of lower depth so this is best case scenario)
  • later, fix feedback/NERO/p&z features first

[aprajita]

please refer to github issue page to see my full checklist of tasks (your email notification probably only contains the first line) from 5 days ago

[anupreeta27]

@aprajita could you say why we need to resize the pngs? is it just for some test purposes or do you mean in general for the AF candidates?

[aprajita]

see screen-grab could be larger

[drphilmarshall]

Agreed, larger images are important. I guess we could make just interpolate our images so that they are 400x400 screen pixels large, but it seems a bit wasteful.

Thanks for checking the feedback messages - I thought they were possible to define, with the training data but obviously not! I'm pretty sure that's something on the Zooniverse to-do list.

For me, the whole question/marker/button interface to the right of the image needs work. When I hit "Done" I expect to get to the next image, it's annoying (and slow) having to click again. And before this, the "Marker" button looks distracting to me. If the task only involves doing one thing, there's no need (and it's better not) to show the button that selects it! Would you agree, PI's?

[aprajita]

Yeah, I hadn't thought of the marker button itself as i just click on the image anyway, but will add it to the list of issues
we certainly can't remove it in the current way the project builder works

i've already asked about how to get rid of the confirmation page and Cam was telling me there's mixed opinions in the zoo team about whether this should appear or not.

i'll get on the other front-end things, including the feedback messages so will also ask about that.

On 17 Nov 2015, at 16:49, Phil Marshall [email protected] wrote:

Agreed, larger images are important. I guess we could make just interpolate our images so that they are 400x400 screen pixels large, but it seems a bit wasteful.

Thanks for checking the feedback messages - I thought they were possible to define, with the training data but obviously not! I'm pretty sure that's something on the Zooniverse to-do list.

For me, the whole question/marker/button interface to the right of the image needs work. When I hit "Done" I expect to get to the next image, it's annoying (and slow) having to click again. And before this, the "Marker" button looks distracting to me. If the task only involves doing one thing, there's no need (and it's better not) to show the button that selects it! Would you agree, PI's?

—
Reply to this email directly or view it on GitHub.

[aprajita]

update
ChrisL said that optionally removing the confirmation page was on their to-do list
also can now ingest gold standard data with feedback. the classifications need to be in the following form. Anu, can you please produce something like this for a handful of test gold standard data?

Gold standard classificaitons should be in the form:

[{
"links": {
"subjects": ["123"]
},
"annotations": [{
task: "T1",
value: 3
}, {
task: "T2",
value: "The value"
}, ...]
}, ...]

[aprajita]

Recent updates

• added Tutorial but now this can't be subject based so we can't do the walk through classification :-(
  tutorial text still needs to be modified
• Field guide added but no content up loaded yet - it's a 1 line perobject format so it's not how we used to have things. we also need to find a way to have the detailed SG available too
• i will need to check about whether we can still have the pop-up messages if users dont login
• there's still much to be done on the layout, image display, marker button and losing the confirmaion screen
• we should think about the different version of subjects we want to upload and whether we want thumbnails to appear around the main image.

[aprajita]

More updates, spoke to Grant, Ali & Cam today, updates in bold
Continue to update this list after discussions subsequently so this list doesnt become too fragmented

SW desirable/essential things for the interface/FrontEnd

• tutorial:
  • no current means of including this (essential) AV to follow-up possibilities for this - tutorial implemented but does not allow interactivity, subjects aren't linked to the tutorial, can only upload snapshots, current tutorial text uploaded without images
  • add images & modify text accordingly
  • Shorten: DES feedback too long
  • change instruction as all clicks are recorded in any image
  • Check with marten is this cumulative & multiple images can it just check if one is consistent with image position
  • add cmd+done & talk to tutorial
• speed:
  • need to remove confirmation screen (essential) AV to issue Is it possible to disable the "Thanks! Your classification" page/step zooniverse/Panoptes-Front-End#1902 is possible to remove this, will ask grant/cam to implement this for us, becomes two buttons, done & done+talk instead of conf screen
  • image load (desirable, needs actual sample data to do speed checks before any query started) need to upload more data
  • image upload speed is slow - how can we speed this up?
• queue up next image:
  • was nice in SW1&2 but not really necessary this is not a feature of panoptes will not happen & i think it's not really important
• pan & zoom
  • was useful particularly for blind search (desirable) check with sample blind & targeted data there is active discussion and ChrisL is not in favour of it. Grant will discuss with him but I am of the feeling given the DES blind images I've seen from Anu it will not be possible to do SW without zoom feature for targeted and pan & zoom for blind. Please comment on this.
• testing feedback
  • more to add when proper example & gold standard data ingested (essential) we should provide a manifest with x,y position, feedback messages for success and non-success and they will take care of putting it in the right format or telling us how to adapt the subject metadata if that's the best way
  • still need to think about best means of identifying the different types of Gold standard data - should this be in the metadata - doesn't hurt to put this in along with success and miss feedback messages as well as providing the manifest. at least the type will be needed for NERO to work i think.
  • have provided marten with feedback manifest, but will need to also change subject to test so that we are consistent with sw1
• instant Feedback
  • will need to be hardcoded for us specifically, was used in snapshot supernova but not a standard feature, supplied message, trianing image tpye, x, y, and radius - note we cannot use the alpha channel not optimal for panoptes
  • from Brian "The feedback feature currently depends on a gold-standard classification being created for a subject. From that we can tell the user "great job" or "keep trying" depending on how close their classification is to the gold standard. SpaceWarps-style feedback (with a popup and knowledge of hits and misses) is not possible at the moment. We had to hardcode the functionality in for Snapshot Supernova. Depending on when SpaceWarps needs it we might have to do the same."
  • note feedback will appear in a fixed position not at a specified x,y, location, as we don't have a confirmation button the message would have to appear after done or done&talk are pressed
• displaying colour settings & loss of quick dashboard
  • investigate how different colours will be shown (different versions accommodated in image upload)
  • investigate how subtracted/unsubtracted images will be shown (different versions accommodated in image upload)
  • investigate possibilities of postage stamps being shown other than play button
  • only options are - play button or side by side (one or the other not both, or users can select), see https://www.zooniverse.org/projects/klmasters/hampshire-landscape-watch/classify
  • Is there a keyboard shortcut to switch between image displays?
  • asked if we can set the multi=plot to the defualt rather than the single image Multiple images: default view zooniverse/Panoptes-Front-End#2708
• changing image classification threshold to NERO input
  • AV liaise with Marten
• back button (not there any more, remove this)

Other things

• data ingestion outside Panoptes Project Builder (currently limited to 10,000 files)
  • AV to investigate best means of uploading data Cam will help with upload, no preferred format just make it available and he will upload Suggest we use the amazon servers method we used before
  • Cam's answer: Yeah, having the data on s3 and providing a CSV manifest (SRC URL, metadata) to us and we can import it. We can help with the s3 transfer too if needed.
• Training subject Frequency
  • will have to accept a single frequency (ratio of trianing images to subject set) NERO will not retire training images and will copy a trianing set from one d`ataset to another
• Subject Zooniverse IDs
  • We will have to randomise IDs before upload (or at least randomise order), the server supplies a sequential ideintifier to data that has been uploaded (if more than one project are uploading at any given time these are interleaved as they come in. Panoptes will give it something of the form e.g. 2056727, we would get a map between the zooniverse IDs and our filenames with the export data. They may allocate a more zooniverse type id but not sure uf that's happening.

Image sizes

• they dont have any feature to automatically scale the images and they leave this up to the browser so we'll have to produced scaled images for display.

NERO output

• Marten will manually give us database each day

OLD Site

• we need to decide how we want to archive the old site but retain access to it

Translations

• not yet implemented but being discussed by developers