Skip to content

Latest commit

 

History

History
119 lines (80 loc) · 3.81 KB

README.en.md

File metadata and controls

119 lines (80 loc) · 3.81 KB
Raw

PoseNet2Scratch

Read this in other languages: English, 日本語

PoseNet2Scratch can detect human pose and add special Scratch blocks that can get x and y position of each part of the body.

It uses PoseNet, a machine learning model , for Real-time Human Pose Estimation. PoseNet advantage is that it can detect human poses quickly and accurately with a normal WebCam without using a special device such as Kinect.

ml5.js, a JavaScript library for machine learning, is used to connect it with Scratch 3.

Try the customized Scratch 3 that has PoseNet2Scratch as an extension.

Examples of use

Requirements

  • OS
    • Windows 8
    • Windows 10
    • MacOS
    • iOS
  • Browser
    • Chrome
    • Safari(iOS)

PoseNet2Scratch sometimes does not work because of some Chrome extensions. Plese switch to Guest Mode in such cases.

How to use

  1. Open https://stretch3.github.io/ on a browser(Google Chrome is recommended). PoseNet2Scratch sometimes does not work because of some Chrome extensions. Plese switch to Guest Mode in such cases.

  2. Click "Add Extension" button(Folder with "+" mark icon).

  1. Select "PoseNet2Scratch" extension.

  1. You can use PoseNet2Scratch blocks.

  1. Select "Video Sensing" extension, if you want to see the image captured by webcam on the stage screen. This is not mandatory.

  1. Add "Glasses" sprite.

  1. Make a code like below.

  1. You will have the glasses shown at the center of your face(x position of the nose) and at the hight of your eyes(y position of the left eye).

With PoseNet2Scratch extension blocks, you can get x and y position of the following body parts:

  • nose
  • left eye
  • right eye
  • left ear
  • right ear
  • left shoulder
  • right shoulder
  • left elbow
  • right elbow
  • left wrist
  • right wrist
  • left hip
  • right hip
  • left knee
  • right knee
  • left ankle
  • right ankle

For Developers - How to run PoseNet2Scratch extension on your computer

  1. Setup LLK/scratch-gui on your computer.

    git clone --depth 1 [email protected]:LLK/scratch-gui.git
cd scratch-gui
npm install

  2. In scratch-gui folder, clone PoseNet2Scratch. You will have posenet2scratch folder under scratch-gui.

    git clone [email protected]:champierre/posenet2scratch.git

  3. Run the install script.

    sh posenet2scratch/install.sh

  4. Run Scratch, then go to http://localhost:8601/.

    npm start

Licence

  • PoseNet2Scratch is under BSD 3-Clause License, open source and freely available to anyone. You can use it at your classes, workshops. Commercial usage is also accepted. If you or your students created something cool using PoseNet2Scratch, please share it on SNS using hashtag #posenet2scratch or let me know to any of these contacts.

Contacts

  • Twitter