Dependency indication in Cameo

About

Since we will maintain downstream chromium/blink for Cameo, may be more other repos in future. It’s necessary to provide a mechanism to indicate the dependency in cameo’s main project repo.

Objective

• The dependency should be accurate as the downstream repo evolving. A bad example of this is to say that cameo dev branch is depends on chromium-cameo lkgr branch. The lkgr branch will be changing overtime, and it will be hard to get the specified source tree back to some history point.
• The indication should come with a solution to check out the whole source tree easily. It will be very helpful for both developer and build infrastructure.
• Cameo should only maintain its real dependency (the downstream repos we forked) instead of copy the whole dependency from chromium.

Solution in chromium

• Chromium has DEPS file in its major project which contains all dependencies of third party repos.
• The dependency is expressed by svn revision or git commit hash id.
• Chromium uses gclient tool to fetch and sync code.
• For versioned checkout, chromium provides the DEPS for specified version individually at http://src.chromium.org/svn/releases/ . It’s a snapshot of all the repos’s (including chromium itself) svn revision.

For cameo

Solution:

Add two deps file into cameo repo, DEPS and DEPS.cameo.
DEPS is like:

vars = {
}

deps = {
}

hooks = [
  {
    # Fetch Cameo dependencies.
    "pattern": ".",
    "action": ["python", "src/cameo/tools/fetch_deps.py", "-v"],
  },
  {
    # A change to a .gyp, .gypi, or to GYP itself should run the generator.
    "pattern": ".",
    "action": ["python", "src/cameo/gyp_cameo"],
  }
]

DEPS.cameo is like:

chromium_version = 'Trunk'
chromium_cameo_point = '45a733d6ac45650d9ddc019ade3c7dbd3b91a816'
deps_cameo = {
  'src': 'ssh://github.com/otcshare/chromium-cameo.git@%s' % chromium_cameo_point,
  'src/third_party/WebKit/Tools/DumpRenderTree': None
}

DEPS.cameo is the one that keeps all cameo’s dependency information. There will be a python script in cameo which will take this DEPS.cameo as input and generate a new .gclient-cameo file for chromium. Then the python script will invoke gclient sync upon it. The .gclient-cameo file will be like:

solutions = [{
  'url': 'ssh://github.com/otcshare/chromium-cameo.git@45a733d6ac45650d9ddc019ade3c7dbd3b91a816',
  'name': 'src',
  'deps_file': '.DEPS.git',
  'custom_deps': {
    'src/third_party/WebKit/Tools/DumpRenderTree': None
  }
}]

And we put the execution of the fetch_deps.py as a hook in cameo’s DEPS file. Which means if we run gclient sync on the cameo project, it will do:

• Checkout cameo
• Run cameo gclient hooks
  • Invoke fetch_deps.py
    • Setup git repo in src/.
      • git init in src/
      • git remote add in src/
      • git fetch in src/
      • git checkout in src/
    • Generate .gclient-cameo.
    • Invoke gclient sync –gclient-file=.gclient-cameo
      • Checkout chromium and its dependencies, which some of them are overwritten if it’s also defined in DEPS.cameo
      • Run chromium gclient hooks
  • Do gyp cameo

For versioned checkout, cameo can do the same thing as chromium. Provide a snapshot that combines the DEPS in chromium and the DEPS.cameo in cameo.

Based on the solution:

• The dependencies is indicated clearly to support bisect.
• Easy to checkout source tree.
  • Gclient config + gclient sync for trunk/version/any history point checkout.
• Integration with our downstream rebase work.
  • Do rebase first. After conflict resolve/build/test/etc, prepare the branch in downstream chromium.
  • Send a pull request to cameo to update DEPS.cameo, maybe some code together to adapt upstream change.
• Integration with cross project development.
  • Developer will first send pull request for chromium/blink part of code.
  • After chromium part got merged, send a pull request to cameo to update DEPS.cameo as well as the cameo part of code.

Compare with CEF3

• The idea is very similar
  • CEF3 uses a text file to indicate the chromium revision it depends on
• The implementation are different:
  • CEF3 use automate.py instead of gclient system for itself, but it do invoke gclient sync for chromium anyway.
  • CEF3 chooses to maintain patches instead of downstream fork, so no custom deps at all.