Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

trial phase #3

Open
10 of 12 tasks
Tracked by #1
serapath opened this issue Apr 16, 2022 · 5 comments
Open
10 of 12 tasks
Tracked by #1

trial phase #3

serapath opened this issue Apr 16, 2022 · 5 comments
Assignees
@serapath

Comments

@serapath
Copy link
Member

serapath commented Apr 16, 2022 •
edited
Loading

@todo 🛠️

@info 👇

dat-ecosystem project input resources 🐝

evaluation of trial phase 📊

  • ❔ did candidates deliver a proper plan to earn the trial phase reward of 💲20 USD 💰
  • ❔ is the plan well structured and understandable by us
    • ❔ is the main issues linked in a comment of the candidate's given trial phase issue for easy discovery by us
    • ❔ are all the issues the candidate made directly or indirectly linked to their main issue
    • ❔ are all issues organized in the style of our github issues project task management system
    • ❔ did the candidate estimate times for their tasks
    • ❔ does the main issue list high level tasks to provide a good overview
    • ❔ are inputs and outputs properly linked
    • ❔ do all inputs lead to outputs which are used in other tasks as inputs until eventually the final outputs are produced
    • ❔ do outputs and inputs fit well together or are there missing steps or missing intermediate output documents
    • ❔ is each task well described in the info section in terms of what will happen or does it leave us with many open questions
    • ❔ did the candidate record a screencast to explain their plan well
  • ❔ does the plan reflect a similar understanding of the dat-ecosystem that we have based on the input resources including 8 years of history
    • ❔ are all trial phase input resources included as task inputs in the candidate's plan
    • ❔ did the candidate provide additional thoughts/notes/links/screenshots/sketches/etc... or any other inspiration for specific parts of the project plan
    • ❔ did the candidate include all the ❓ outputs mentioned in the trial phase issue or adapted them to something similarly useful
      • the outputs mentioned were:
        • @output ❓ website UI/UX specification & wireframes
        • @output ❓ dat-ecosystem design system
        • @output ❓ list of re-usable web components
        • @output ❓ implemented website
        • @output ❓ link to one or multiple final screencasts (each 3-5 minutes max) where the candidate walks us through the entire project plan
    • ❔ does the proposed website structure represent all aspects of the dat-ecosystem input resources
  • ❔ did the candidate work with us during the trial phase and used our feedback to improve the plan
    • ❔ did they use worklog issue comments to post their questions
    • ❔ did they improve the plan structure based on our feedback
    • ❔ did the candidate mention time spent working on the task when posting a worklog
  • ❔ does it seem as if the plan can be executed in 4-8 weeks
This was referenced Apr 16, 2022
Open
Closed
@serapath serapath self-assigned this Apr 16, 2022
@serapath serapath mentioned this issue Apr 17, 2022
Closed
6 tasks
This was referenced Apr 25, 2022
Closed
Closed
Closed
@serapath
Copy link
Member Author

Checklist

ABSTRACT EXAMPLE: https://hackmd.io/wTuH5UPLRdi214RDjRwWGQ?view

CHECKLIST: https://hackmd.io/OzSDw7a6ShS78ZS6ITnZuA?view

  1. DISCORD: We use discord chat about our work on projects
  • every time we finish a work effort at the end of our day or sometimes in between
  • we post the latest worklog video links to the worklog chat on discord
  1. ISSUES: creating a new issue can be done with the "new issue" button
  • but also by clicking on a textual todo in the todolist can convert it into a sub issue
  • no issue should have more than 10-20 todos
  • if a todo list grows larger, some todos need to be grouped and put into a linked sub issue
  • a sub issue todo can be marked as done only if all todos inside the sub issues are marked as done
  1. ISSUE @todo: each project has a main issue with a task list and info section
  • the task list shows all open and finished high level todos
  • next to every todo you can link input documents and/or output documents
  • some tasks can be links to sub issue with sub task lists and inputs/outputs
  • the comments contain worklog comments and feedback comments
  • issues can be assigned to you or others to indicate who is working on the task lists
  • a todo should only be started once all inputs are marked as finished (= 📦)

  1. ISSUE @INFO: section can describe an issue with todos in more detail

  2. RESULTS: each todo can have multiple inputs links or outputs links

  • each output link references not an issue, but a document
    • e.g. (files/links/wireframes/moodboards/documents/...)
  • the symbol ❓ result name means an input or output is planned
    • result name is not a link
  • the symbol 🏭 result name means an input or output is work in progress
    • result name is a link to a final or temporary document
  • the symbol 📦 result name means an input or output is done
    • result name is a link to a final document added to the repository
  • a worklog introduces an output
  • the output is then linked under the finished todo that produced the output
  • the suffix from [comment](link to worklog comment) is added to the output
  • a sub bullet point list can list other issues where the output is used as input
    • just add one bullet point per usage as input like: next #<issue number>
  • an input document link in an issue needs a suffix (from link to issue that produced the output)

  1. WORKLOGS: After every work effort, we record a screencast worklog video

    • publish the screencast worklog video to youtube
    • add links to all github issues mentioned to the video description
    • post the video link in a worklog comment in all related github issues
    • link all produced finished or work-in-progress results in that issue too
    • add our video to our personal worklog playlist on youtube
    • also link all outputs next to the finished todos which produced them

  2. FEEDBACKS: We will watch results and videos and write a feedback comment

    • the feedback might contain more requests to create and refine existing todos
    • the feddback can also answer questions from your worklog video
    • a feedback issue might contain task labels
      • which means you should take the feedback and create or update todos from the feedback

  3. STRUCTURE: we try to structure all issue and sub issue todos lists

    • by components/parts and inputs/outputs
    • sometimes this can also include phases
    • structuring an issue so it helps with overview is an art form and comes with practice
    • the goal is to help all of us to keep track of progress and all results
    • the initial main issue is for overview
    • each sub issue is for a group of tasks that belong to a part of the project

  4. WORK: working with issues means to constantly update and change the structure a bit

    • e.g. creating or updating issues and todo lists
    • e.g. marking todos as finished
    • e.g. linking more inputs and outputs or change their status

@Hornet004 Hornet004 mentioned this issue Apr 26, 2022
Closed
4 tasks
This was referenced Apr 26, 2022
Closed
Closed
Closed
This was referenced Apr 27, 2022
Closed
Closed
Closed
Closed
Closed
@CodeLawd CodeLawd mentioned this issue Apr 28, 2022
Closed
3 tasks
@serapath
Copy link
Member Author

serapath commented May 1, 2022 •
edited
Loading

@CodeLawd evaluation of trial phase 📊

  • ✔️ did candidates deliver a proper plan to earn the trial phase reward of 💲20 USD 💰
  • ✔️ is the plan well structured and understandable by us
    • ✔️ is the main issues linked in a comment of the candidate's given trial phase issue for easy discovery by us
    • ❌ are all the issues the candidate made directly or indirectly linked to their main issue
    • ✔️ are all issues organized in the style of our github issues project task management system
      • ✔️ all issues have @todo and @info section
      • ❌ all todos are bullet points in the @todo section and have checkboxes and are either textual or issue links
      • ✔️ all @inputs have an icon and name and indicate the issue they came from via from #<issueno>
      • ❌ all @outputs are either final 🏁 or are inputs into other todos and mark those with next #<issueno>
    • ✔️ did the candidate estimate times for their tasks
    • ✔️ does the main issue list high level tasks to provide a good overview
    • ❌ do all inputs lead to outputs which are used in other tasks as inputs until eventually the final outputs are produced
    • ❌ do outputs and inputs fit well together or are there missing steps or missing intermediate output documents
      • we concluded that many outputs, even after reading the @info section were not perfectly clear to us in terms of what exactly to expect, because they have not examples or description attached that would explain it exactly
    • ❌ is each task well described in the info section in terms of what will happen or does it leave us with many open questions
      • it leaves us with many questions, e.g. issue Design Brief  #38 spends 4 hours on reading input documents, but then 3 days on producing a design brief document which is not used as input and it is unclear what it contains and the same is true for many other inputs and we prefer minimalism - so what is not absolutely required should not be produced to save time and resources
    • ❌ did the candidate record a screencast to explain their plan well
  • ❌ does the plan reflect a similar understanding of the dat-ecosystem that we have based on the input resources including 8 years of history
    • the plan does not reflect our understanding of dat-ecosystem because most input resources where not used in the plan and it includes a competitive analysis which was not mentioned in input documents and is out of scope for the project
    • ❌ are all trial phase input resources included as task inputs in the candidate's plan
      • only a few input resources are selected and it is unclear why not all or why those that have been selected
      • issue Design Brief  #38 includes videos as input that were not inputs for the dat-ecosystem website but only input for the trial phase and should have been watched before doing the trial phase
    • ❌ did the candidate provide additional thoughts/notes/links/screenshots/sketches/etc... or any other inspiration for specific parts of the project plan
      • no links to resources or any inputs beyond minimal text in the @info section has been provided
    • ❌ did the candidate include all the ❓ outputs mentioned in the trial phase issue or adapted them to something similarly useful
      • the outputs mentioned were:
        • @output ❓ website UI/UX specification & wireframes
        • @output ❓ dat-ecosystem design system
        • @output ❓ list of re-usable web components
        • @output ❓ implemented website
        • @output ❓ link to one or multiple final screencasts (each 3-5 minutes max) where the candidate walks us through the entire project plan
    • ❌ does the proposed website structure represent all aspects of the dat-ecosystem input resources
      • no first rough draft of website structure has been proposed in form of planned outputs or todos
  • ❔ did the candidate work with us during the trial phase and used our feedback to improve the plan
    • ❌ did they use worklog issue comments to post their questions
    • ✔️ did they improve the plan structure based on our feedback
    • ❌ did the candidate mention time spent working on the task when posting a worklog
  • ✔️ does it seem as if the plan can be executed in 4-8 weeks

Sadly the final worklog comment had a list of all outputs in the entire project, which is not how we do it and it was not necessary, but it was lacking the screencast to walk us through your pllan and it was lacking the total time it took you to finish everything.

other notes

- trial phase issue worklog comment
    - PLUS: good comment and linking
    - MINUS:
- main issue
    - PLUS: 8 phases, estimation
    - MINUS:
- design brief
    - PLUS: 
    - MINUS: weird estimation, not clear what is brief document and why it takes 3 days to make it
- requirement gathering
    - PLUS: 
    - MINUS: not usign brief document, product requirement section is weird, logic not good
- conceptual design
    - PLUS: 
    - MINUS:
- ui prototyping
    - PLUS: lots of wireframing
    - MINUS:
- web dev
- testing and feedback
    - PLUS: good to think about testing with potencial users and get feedbacks
- refinment

@serapath
Copy link
Member Author

serapath commented May 1, 2022 •
edited
Loading

@Helenphina evaluation of trial phase 📊

  • ✔️ did candidates deliver a proper plan to earn the trial phase reward of 💲20 USD 💰
  • ✔️ is the plan well structured and understandable by us
    • ❌ is the main issues linked in a comment of the candidate's given trial phase issue for easy discovery by us
    • ✔️ are all the issues the candidate made directly or indirectly linked to their main issue
    • ✔️ are all issues organized in the style of our github issues project task management system
    • ✔️ did the candidate estimate times for their tasks
    • ✔️ does the main issue list high level tasks to provide a good overview
    • ✔️ do all inputs lead to outputs which are used in other tasks as inputs until eventually the final outputs are produced
    • ✔️ do outputs and inputs fit well together or are there missing steps or missing intermediate output documents
    • ✔️ is each task well described in the info section in terms of what will happen or does it leave us with many open questions
    • ❌ did the candidate record a screencast to explain their plan well
  • ❌ does the plan reflect a similar understanding of the dat-ecosystem that we have based on the input resources including 8 years of history
    • ❌ are all trial phase input resources included as task inputs in the candidate's plan
    • ✔️ did the candidate provide additional thoughts/notes/links/screenshots/sketches/etc... or any other inspiration for specific parts of the project plan
    • ✔️ did the candidate include all the ❓ outputs mentioned in the trial phase issue or adapted them to something similarly useful
      • the outputs mentioned were:
        • @output ❓ website UI/UX specification & wireframes
        • @output ❓ dat-ecosystem design system
        • @output ❓ list of re-usable web components
        • @output ❓ implemented website
        • @output ❓ link to one or multiple final screencasts (each 3-5 minutes max) where the candidate walks us through the entire project plan
    • ❌ does the proposed website structure represent all aspects of the dat-ecosystem input resources
      • sadly the input resources have not been processed to already suggest a first draft of todos for parts of the website
  • ✔️ did the candidate work with us during the trial phase and used our feedback to improve the plan
    • ✔️ did they use worklog issue comments to post their questions
    • ✔️ did they improve the plan structure based on our feedback
    • ❌ did the candidate mention time spent working on the task when posting a worklog
  • ✔️ does it seem as if the plan can be executed in 4-8 weeks

Sadly the final worklog comment had a list of all outputs in the entire project, which is not how we do it and it was not necessary, but it was lacking the screencast to walk us through your pllan and it was lacking the total time it took you to finish everything.

other notes

- trial phase issue worklog comment
    - MINUS: no booked time, no link to the main issue, listed all the outputs, no screencast
- main issue:
    - PLUS: 
    - MINUS: it doesn explain what each phase means and from titles we can't know for sure
- project brief
    - PLUS: added estimates, linked and gorupsed the resources, output
    - MINUS: included resources that were for this task
- ux survey
    - PLUS:good ideas, understands the theory of communication/marketing, added links to the theory to help us understand the phases
    - MINUS:
- design systems
    - PLUS: ok, always estimating
    - MINUS:
- ui design
    - PLUS: ok
    - MINUS

@serapath
Copy link
Member Author

serapath commented May 1, 2022 •
edited
Loading

@Hornet004 evaluation of trial phase 📊

  • ✔️ did candidates deliver a proper plan to earn the trial phase reward of 💲20 USD 💰
  • ❌ is the plan well structured and understandable by us
    • the plan is very simple and generic with not much detail so its unclear what is happening in detail
    • ❌ is the main issues linked in a comment of the candidate's given trial phase issue for easy discovery by us
    • ❌ are all the issues the candidate made directly or indirectly linked to their main issue
    • ✔️ are all issues organized in the style of our github issues project task management system
      • even though there are some minor missing parts that were mentioned in the feedback but not processed
    • ❌ did the candidate estimate times for their tasks
      • ❌ do all tasks have estimations
    • ✔️ does the main issue list high level tasks to provide a good overview
    • ❌ do all inputs lead to outputs which are used in other tasks as inputs until eventually the final outputs are produced
      • issue Web UI Development #16 does not have wireframes/iterations and ui designs as input, but final figma ui designs for website which wasnt mentioned before. so maybe that is the same, but because of the different naming it is not clear
      • also there are unlinked but not closed issues that are misstructured too (see the list above)
    • ❌ do outputs and inputs fit well together or are there missing steps or missing intermediate output documents
      • the web ui development just converts the figma files in one step into the website with not a lot of details
      • the product design turns the vaguely named solution concept into wireframes in one go with no details
      • the information architecture turns all the inputs in one step into the solution concept
      • it is not clear what the process is and therefor also hard to estimate for us what is happening and how long it might take
    • ✔️ is each task well described in the info section in terms of what will happen or does it leave us with many open questions
      • there is in fact some additional information, but a lot of it could have instead been presented in the todos and inputs/outputs just by naming things better and giving more details, so to us it feels the info section is/was used to avoid specifying more information by a better structured todo section
    • ✔️ did the candidate record a screencast to explain their plan well
  • ❌ does the plan reflect a similar understanding of the dat-ecosystem that we have based on the input resources including 8 years of history
    • we expected a lot more details and it is absolutely generic instead
    • ❌ are all trial phase input resources included as task inputs in the candidate's plan
    • ❌ did the candidate provide additional thoughts/notes/links/screenshots/sketches/etc... or any other inspiration for specific parts of the project plan
    • ❌ did the candidate include all the ❓ outputs mentioned in the trial phase issue or adapted them to something similarly useful
      • the outputs mentioned were:
        • @output ❓ website UI/UX specification & wireframes
        • @output ❓ dat-ecosystem design system
        • @output ❓ list of re-usable web components
        • @output ❓ implemented website
        • @output ❓ link to one or multiple final screencasts (each 3-5 minutes max) where the candidate walks us through the entire project plan
    • ❌ does the proposed website structure represent all aspects of the dat-ecosystem input resources
  • ✔️ did the candidate work with us during the trial phase and used our feedback to improve the plan
    • ✔️ did they use worklog issue comments to post their questions
    • ✔️ did they improve the plan structure based on our feedback
    • ✔️ did the candidate mention time spent working on the task when posting a worklog
  • ❔ does it seem as if the plan can be executed in 4-8 weeks

Even though the candidate did communicate the most and we gave the most feedback out of all candidates, we are a bit surprised that the plan is quite simple and not structured with more details.

  • A special 🏆 for the creativity of creating a task new dat-ecosystem website project plan #12 to work on the plan itself. This makes a lot of sense and others should have done it to
  • A special 🏆 for being the only one that recorded the required screencast at the end to present the work.
    • here again a document is shown that is or was not plan of the github issue, so the same as we felt the @info section was used were the @todo section could have been used, the document with the MIRO tool served as something that should instead have been covered by the github issues, which makes us feel the candidate avoids using out tooling, which makes it hard for us to work together, because our system absolutely relies on strict usage of our github based remote coworking flows.
  • A lst 🏆 goes for posting the spent time on the final worklog comment (8h) which also no other candidate did

other notes

- trial phase issue
    - PLUS: booked time, linked screencast
    - MINUS:, didn't link his main issue linked outputs that don't belong here
- main issue:
    - PLUS: correctly linked screencast as output, made himself a roadmap issue, added an info section with the brief, lots of comment bouncing and listening to feedbacks
    - MINUS: no link to the main issue
- main plan issue:
    - PLUS: ok and relatively clear main issues, good he added the info section
    - MINUS: bad naming for main 3 phases, no time estimations and no booked times anymore, 
- information architecture
    - PLUS: good linking of inputs and outputs, 
- product design
    - PLUS: 
    - MINUS: not clear what happens in product design phase, too low resolution (hard to estimate how long tasks will take), very generic
- web UI design
    - PLUS:
    - MINUS: quite shallow, just says - implementing whatever came from the wireframes, not thinking in components

@serapath
Copy link
Member Author

serapath commented May 1, 2022

EVALUATION

The most outstanding job that was very strictly adhering to the trial phase description and covered all requirements was the work done by @Helenphina. Both of the other candidates had different deficits compared to that.
A main problem with @CodeLawd was that we had almost no opportunity to directly interact and give feedback, because we feel that way many minor issues could have been improved and solved. The times and deadlines were clear before the trial phase started, so this didn't make things super reliable, but the delivered results are still ok.
With @Hornet004 as a candidate we were surprised that a lot of feedback and even work on additional issues was not used in the end. It seems though a lot of work went into documents shown in the screencast and in parts in additional description in the @info section, but could have went into better compliance with the task system, because getting to know the task system and how we work and practicing this for a few days was part of the task.

Candidates:

Expectations:

  1. We were expecting the candidates to work with us for a few days and post worklogs and questions and use the feedback
  2. We were expecting the candidates to absolutely strictly stick to the github project issue management system and learn it
  3. We were expecting the detailed plan to basically contain ALL of the input resources listed in the trial phase

Regarding 1. and 2. all candidates delivered to some degree (see also evaluation results)
Regarding 3. we are not really satisfied.

NEXT STEPS

  1. We hope the trial phase was ok for everyone and we are happy for feedback to learn how we can improve
  2. We think that everyone did a good job to the degree that we pay 20 USD to every candidate (please let us know how)
  3. We need an additional day or two to set up the final project plan for the next phase, because none of the existing plans covers the project well enough yet for out taste.

Some candidates did a lot better than others but I will tomorrow work on a project plan for the entire project and then see if we can (based on strength and weaknesses) work with more than one candidate to proceed and get some work done in parallel.
I will inform every candidate either monday or tuesday to share the project plan and whether they have been selected.

Thank you everyone and we hope the overall experience was fine, but please let us know in detail what you liked and what you did not like so we can improve in the future :-)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@serapath serapath

Labels
None yet
Milestone
No milestone
Development

No branches or pull requests
1 participant
@serapath