添加保存过程中产生的数据文件到github action cache选项，顺便重新整理了一下github aciton workflow (#…

…464)

* 添加保存过程中产生的数据文件到github action cache选项，顺便重新整理了一下github aciton workflow

1. run_data_sync 调用 gh_page方式改成了workflow_call，这样可以传递一些数据过去。可以把设置都放在run_data_sync env中完成。在github后台看来也在一个job中完成了。
2. 缓存默认关闭，添加了文档。
3. 三个workflow中的action版本做了升级。有好几个github action提示版本过低了。
4. python和node准备环境部分做了优化。用了setup python /setup node自带的cache参数。多余的几个step删了。

* node-version: "16" -> node-version: '16'

* feat: add BUILD_GH_PAGES to control build gh pages

* docs: BUILD_GH_PAGES in run_data_sync.yml

.github/workflows/ci.yml

@@ -11,25 +11,18 @@ jobs:
         python-version: ['3.8', '3.9', '3.10']
     steps:
       - uses: actions/checkout@v3
+
       - name: Set up Python ${{ matrix.python-version }}
+        id: setup_python
         uses: actions/setup-python@v4
         with:
           python-version: ${{ matrix.python-version }}
-      # from pdm
-      - name: Set Variables and install dep
-        id: set_variables
-        run: |
-          sudo apt-get install libxml2-dev libxslt-dev python3-dev 
-          echo "PY=$(python -c 'import hashlib, sys;print(hashlib.sha256(sys.version.encode()+sys.executable.encode()).hexdigest())')" >> $GITHUB_OUTPUT
-          echo "PIP_CACHE=$(pip cache dir)" >> $GITHUB_OUTPUT
-      - name: Cache PIP
-        uses: actions/cache@v2
-        with:
-          path: ${{ steps.set_variables.outputs.PIP_CACHE }}
-          key: ${{ runner.os }}-pip-${{ steps.set_variables.outputs.PY }}
+          cache: 'pip'
+          cache-dependency-path: 'requirements-dev.txt'
+
       - name: Install dependencies
         run: |
-          python -m pip install --upgrade pip
           pip install -r requirements-dev.txt
+
       - name: Check formatting (black)
         run: black . --check

.github/workflows/gh-pages.yml

@@ -2,14 +2,26 @@ name: Publish GitHub Pages
 
 # Controls when the action will run.
 on:
-  # trigger when the Run Data Sync workflow succeeded
-  workflow_run:
-    workflows: ["Run Data Sync"]
-    types:
-      - completed
-
-  # Allows you to run this workflow manually from the Actions tab
   workflow_dispatch:
+    inputs:
+      save_data_in_github_cache:
+        required: true
+        type: boolean
+        default: false
+      data_cache_prefix:
+        required: true
+        type: string
+        default: track_data
+  workflow_call:
+    inputs:
+      save_data_in_github_cache:
+        required: true
+        type: boolean
+        default: false
+      data_cache_prefix:
+        required: true
+        type: string
+        default: track_data
 
 # A workflow run is made up of one or more jobs that can run sequentially or in parallel
 jobs:
@@ -21,34 +33,43 @@ jobs:
     # Steps represent a sequence of tasks that will be executed as part of the job
     steps:
       # Checks-out your repository under $GITHUB_WORKSPACE, so your job can access it
-      - uses: actions/checkout@v2
+      - uses: actions/checkout@v3
 
-      - name: Setup Node.js environment
-        uses: actions/[email protected]
+      - name: Cache Data Files
+        if: inputs.save_data_in_github_cache
+        uses: actions/cache@v3
         with:
-          node-version: "16"
-
-      - name: Get yarn cache directory path
-        id: yarn-cache-dir-path
-        run: echo "dir=$(yarn cache dir)" >> $GITHUB_OUTPUT
+          path: |
+            activities
+            assets
+            GPX_OUT
+            TCX_OUT
+            FIT_OUT
+            Workouts
+            scripts/data.db
+            src/static/activities.json
+            imported.json
+          key: ${{ inputs.data_cache_prefix }}-${{ github.sha }}-${{ github.run_id }}
+          restore-keys: |
+            ${{ inputs.data_cache_prefix }}-${{ github.sha }}-
+            ${{ inputs.data_cache_prefix }}-
 
-      - uses: actions/cache@v2
-        id: yarn-cache # use this to check for `cache-hit` (`steps.yarn-cache.outputs.cache-hit != 'true'`)
+      - name: Setup Node.js environment
+        uses: actions/setup-node@v3
         with:
-          path: ${{ steps.yarn-cache-dir-path.outputs.dir }}
-          key: ${{ runner.os }}-yarn-${{ hashFiles('**/yarn.lock') }}
-          restore-keys: |
-            ${{ runner.os }}-yarn-
+          node-version: '16'
+          cache: 'yarn'
 
       - run: |
           yarn install
+
       - run: |
           PATH_PREFIX=/${{ github.event.repository.name }} yarn build --prefix-paths
       - run: |
           cp -r assets/ public/static
 
       - name: GitHub Pages
-        uses: crazy-max/ghaction-github-pages@v2.2.0
+        uses: crazy-max/ghaction-github-pages@v3
         with:
           # Git branch where site will be deployed
           target_branch: gh-pages

.github/workflows/run_data_sync.yml

@@ -21,7 +21,7 @@ on:
 
 env:
   # please change to your own config.
-  RUN_TYPE: pass # support strava/nike/garmin/garmin_cn/keep/only_gpx/nike_to_strava/strava_to_garmin/strava_to_garmin_cn/garmin_to_strava/garmin_to_strava_cn/codoon, Please change the 'pass' it to your own
+  RUN_TYPE: pass # support strava/nike/garmin/garmin_cn/keep/only_gpx/only_fit/nike_to_strava/strava_to_garmin/strava_to_garmin_cn/garmin_to_strava/garmin_to_strava_cn/codoon, Please change the 'pass' it to your own
   ATHLETE: yihong0618
   TITLE: Yihong0618 Running
   MIN_GRID_DISTANCE: 10 # change min distance here
@@ -34,37 +34,52 @@ env:
   # Dont making this huge, just picking points you needing. https://developers.google.com/maps/documentation/utilities/polylineutility using this tool to making your polyline
   IGNORE_POLYLINE: "ktjrFoemeU~IorGq}DeB"
   IGNORE_RANGE: 10 # Unit meter
+  SAVE_DATA_IN_GITHUB_CACHE: false # if you deploy in the vercal, check the README
+  DATA_CACHE_PREFIX: "track_data"
+  BUILD_GH_PAGES: false
 
 jobs:
   sync:
     name: Sync
     runs-on: ubuntu-latest
+    outputs:
+      SAVE_DATA_IN_GITHUB_CACHE: ${{ steps.set_output.outputs.SAVE_DATA_IN_GITHUB_CACHE }}
+      DATA_CACHE_PREFIX: ${{ steps.set_output.outputs.DATA_CACHE_PREFIX }}
+      BUILD_GH_PAGES: ${{ steps.set_output.outputs.BUILD_GH_PAGES }
     steps:
       - name: Checkout
         uses: actions/checkout@v3
+
       - name: Set up Python
+        id: setup_python
         uses: actions/setup-python@v4
         with:
-          python-version: 3.8
-
-      # from pdm and install dep
-      - name: Set Variables
-        id: set_variables
-        run: |
-          sudo apt-get install libxml2-dev libxslt-dev python3-dev 
-          echo "PY=$(python -c 'import hashlib, sys;print(hashlib.sha256(sys.version.encode()+sys.executable.encode()).hexdigest())')" >> $GITHUB_OUTPUT
-          echo "PIP_CACHE=$(pip cache dir)" >> $GITHUB_OUTPUT
+          python-version: "3.10"
+          cache: pip
+          cache-dependency-path: 'requirements.txt'
 
-      - name: Cache PIP
-        uses: actions/cache@v2
-        with:
-          path: ${{ steps.set_variables.outputs.PIP_CACHE }}
-          key: Ubuntu-pip-${{ steps.set_variables.outputs.PY }}
       - name: Install dependencies
         run: |
-          python -m pip install --upgrade pip
           pip install -r requirements.txt
-        if: steps.pip-cache.outputs.cache-hit != 'true'
+
+      - name: Cache Data Files
+        if: env.SAVE_DATA_IN_GITHUB_CACHE == 'true'
+        uses: actions/cache@v3
+        with:
+          path: |
+            activities
+            assets
+            GPX_OUT
+            TCX_OUT
+            FIT_OUT
+            Workouts
+            scripts/data.db
+            src/static/activities.json
+            imported.json
+          key: ${{ env.DATA_CACHE_PREFIX }}-${{ github.sha }}-${{ github.run_id }}
+          restore-keys: |
+            ${{ env.DATA_CACHE_PREFIX }}-${{ github.sha }}-
+            ${{ env.DATA_CACHE_PREFIX }}-
 
       - name: Run sync Nike script
         if: env.RUN_TYPE == 'nike'
@@ -112,6 +127,16 @@ jobs:
         run: |
           python scripts/gpx_sync.py
 
+      - name: Run sync Only FIT script
+        if: env.RUN_TYPE == 'only_fit'
+        run: |
+          python scripts/fit_sync.py
+
+      - name: Run sync Only TCX script
+        if: env.RUN_TYPE == 'only_tcx'
+        run: |
+          python scripts/tcx_sync.py
+
       - name: Run sync Strava to Garmin(Run with strava(or others upload to strava) data backup in Garmin)
         if: env.RUN_TYPE == 'strava_to_garmin'
         run: |
@@ -146,9 +171,27 @@ jobs:
           python scripts/gen_svg.py --from-db --year $(date +"%Y")  --language zh_CN --title "$(date +"%Y") Running" --type github --athlete "${{ env.ATHLETE }}" --special-distance 10 --special-distance2 20 --special-color yellow --special-color2 red --output assets/github_$(date +"%Y").svg --use-localtime --min-distance 0.5
 
       - name: Push new runs
+        if: env.SAVE_DATA_IN_GITHUB_CACHE != 'true'
         run: |
           git config --local user.email "${{ env.GITHUB_EMAIL }}"
           git config --local user.name "${{ env.GITHUB_NAME }}"
           git add .
           git commit -m 'update new runs' || echo "nothing to commit"
           git push || echo "nothing to push"
+
+      - name: Set Output
+        id: set_output
+        run: |
+          echo "SAVE_DATA_IN_GITHUB_CACHE=${{ env.SAVE_DATA_IN_GITHUB_CACHE }}" >> "$GITHUB_OUTPUT"
+          echo "DATA_CACHE_PREFIX=${{ env.DATA_CACHE_PREFIX }}" >> "$GITHUB_OUTPUT"
+          echo "BUILD_GH_PAGES=${{ env.BUILD_GH_PAGES }}" >> "$GITHUB_OUTPUT"
+
+  publish_github_pages:
+    if: needs.sync.result == 'success' && needs.sync.outputs.BUILD_GH_PAGES == 'true'
+    name: Build and publish Github Pages
+    uses: ./.github/workflows/gh-pages.yml
+    with:
+      save_data_in_github_cache: ${{needs.sync.outputs.SAVE_DATA_IN_GITHUB_CACHE == 'true'}}
+      data_cache_prefix: ${{needs.sync.outputs.DATA_CACHE_PREFIX}}
+    needs:
+      - sync

README-CN.md

@@ -823,13 +823,38 @@ Actions [源码](https://github.com/yihong0618/running_page/blob/master/.github/
 
 1. 更改成你的 app type 及 info
    ![image](https://user-images.githubusercontent.com/15976103/94450124-73f98800-01df-11eb-9b3c-ac1a6224f46f.png)
+
 2. 在 repo Settings > Secrets 中增加你的 secret (只添加你需要的即可)
+
    ![image](https://user-images.githubusercontent.com/15976103/94450295-aacf9e00-01df-11eb-80b7-a92b9cd1461e.png)
    我的 secret 如下
    ![image](https://user-images.githubusercontent.com/15976103/94451037-8922e680-01e0-11eb-9bb9-729f0eadcdb7.png)
 
 </details>
 
+## 把数据文件放在github cache中
+
+<details>
+<summary>把数据文件放在github cache中</summary>
+
+`run_data_sync.yml`中的`SAVE_DATA_IN_GITHUB_CACHE`设置为`true`时，可以把脚本抓取和中间产生的数据文件放到github action cache中。这样可以让你的github commit历史和目录保持干净。
+
+如果你用github pages部署建议把这个值设置成`true`。
+
+如果你使用`Vervel`，需要在vercel后台进行下面几步操作：
+
+1. 在`General`的`Build & Development Settings`的Framework Preset选择Other，然后把override全部打开，命令行内容全部设置成空。
+
+  ![General](https://user-images.githubusercontent.com/413855/255507920-76932fb9-61d6-423e-92e2-2e92b5bbd432.png)
+
+2. 在Git模块中，`Production Branch`的Branch name填入`gh-pages`
+
+  ![Branch name](https://user-images.githubusercontent.com/413855/255508137-ef742815-8418-419a-86b4-6756ff68e1db.png)
+
+3. `run_data_sync.yml`中的`BUILD_GH_PAGES`设置为`true`
+
+</details>
+
 # Fit 文件
 
 测试发现，不同厂商在写fit文件的时候有略微差异。

README.md

@@ -672,6 +672,31 @@ The following steps need to be taken
 
 </details>
 
+## Storing Data Files in GitHub Cache
+
+<details>
+<summary>Storing Data Files in GitHub Cache</summary>
+
+When `SAVE_DATA_IN_GITHUB_CACHE` is set to `true` in the `run_data_sync.yml` file, the script can store fetched and intermediate data files in the GitHub Action cache. This helps keep your GitHub commit history and directory clean.
+
+If you are deploying using GitHub Pages, it is recommended to set this value to `true`, and set  `BUILD_GH_PAGES` to true.
+
+If you are using `Vercel`, you need to perform the following steps in the Vercel dashboard:
+
+1. In the `General` section of `Build & Development Settings`, choose `Other` as the `Framework Preset`, and then enable all the overrides, leaving the command line content empty.
+
+  ![General](https://user-images.githubusercontent.com/413855/255507920-76932fb9-61d6-423e-92e2-2e92b5bbd432.png)
+
+2. In the Git module, enter `gh-pages` as the `Production Branch` Branch name.
+
+  ![Branch name](https://user-images.githubusercontent.com/413855/255508137-ef742815-8418-419a-86b4-6756ff68e1db.png)
+
+
+3. Set the `BUILD_GH_PAGES` to true, in `run_data_sync.yml`.
+
+</details>
+
+
 # Fit file
 
 supported manufacturer：