Detect new brands: CPDEVICE, COLORROOM, CMF, E-TACHI, Korax, XPPen an…

…d Detect devices for exist brands (#205) feat(sync): Adds detection for Blackbox Exporter bot (matomo-org#7842) --- feat(sync) Detect new brand COLORROOM And Detect devices for exist brands (matomo-org#7848) feat(device) detect brand Hotwav: Hyper 7 Pro feat(device) detect brand Realme: 13 Pro 5G (RMX3988) feat(device) detect brand Vivo: V17 feat(device) detect brand Shivaki: STV-55LED41 feat(device) detect brand OPPO: A3x 5G (PKD130) feat(device) detect brand NUU Mobile: B30 Pro (S6702X) feat(device) detect brand Vivo: Y03t (V2409, V2344), V40 (V2348) feat(device) detect new brand COLORROOM: K10C feat(device) detect brand Google: Pixel 9 Pro, Pixel 9, Pixel 9 Pro Fold feat(device) detect brand Motorola: Moto Razr 50 feat(device) detect brand Condor: Nova 60 feat(device) detect brand EXCEED: EX8S1 feat(device) detect brand ZTE: Fresh 50 (7543N) feat(device) detect brand LT Mobile: S33 feat(device) detect brand Doogee: Blade 10 Ultra, Blade 10 Pro, Blade 10 feat(device) detect brand CUBOT: King Kong ES feat(device) detect brand Huawei: Honor X6b (JDY-LX2) feat(device) detect brand Tecno Mobile: Camon 19 feat(device) detect brand Realme: 12 5G (RMX3992), 13 Pro 5G (RMX3990), Narzo N61 (RMX3933), 13 Pro+ 5G (RMX3921), Narzo 70 5G (RMX3869) feat(device) detect brand Nubia: Focus Pro 5G (Z2351N) feat(device) detect brand Dcode: Cygnal 4 Lite (DS-CL4 Lite) feat(device) detect brand Starwind: SW-LED65UG402, SW-LED43UB400, SW-LED43SB300, SW-LED40SB300, SW-LED32SB304 --- feat(sync) Improves detection for Aloha Browser family (matomo-org#7850) --- feat(sync) Detect new brand E-TACHI And Detect devices for exist brands (matomo-org#7851) feat(device) detect new brand E-TACHI: A5 Slim feat(device) detect brand Nothing Phone: 1 (A015) feat(device) detect brand Xiaomi: 14T Pro (2407FPN8EG) feat(device) detect brand Sharp: Aquos V6 (SH-C03), Aquos V6+ (SH-C04) feat(device) detect brand Dcode: Cypher (DS-CR1) feat(device) detect brand Wiko: Y51 (W-K211-OPE) feat(device) detect brand OnePlus: Pad 2 12.1" WiFi (OPD2403) feat(device) detect brand OPPO: Pad Neo (OPD2303) feat(device) detect brand Hometech: 7 Premium Pro feat(device) detect brand Skyline: 50UST5970 feat(device) detect brand Asano: 50LU8130S 50.0", 50LF7010T 50.0" feat(device) detect brand ZTE: Blade A35 (Z2453) feat(device) detect brand Motorola: Moto X50 Ultra (XT2401-2) feat(device) detect brand VOCAL: V11 feat(device) detect brand Huawei: Honor Magic V3 (FCP-N49) feat(device) detect brand Oukitel: WP39, WP28 S, WP28 E feat(device) detect brand HTC: U Play feat(device) detect brand Nothing Phone: 2a feat(device) detect brand Hammer: Iron Va feat(device) detect brand AllDocube: iPlay 60 Mini Pro feat(device) detect brand Energizer: Energy S550 feat(device) detect brand Tecno Mobile: Spark 30C (TECNO KL5), Spark Go (2022) (TECNO KG5m) feat(device) detect brand Sparx: Ultra 8 feat(device) rename brand Nothing Phone to Nothing and rename device models feat(device) detect brand CMF: Phone 1 (A015) --- feat(sync) Improves detection for client hints fragment (matomo-org#7852) --- feat(sync) Detect new brands: CPDEVICE, Korax, XPPen and Detect devices for exist brands (matomo-org#7855) feat(device) detect brand Google: Pixel 4a (5G) (G025E) feat(device) detect brand OPPO: A3 5G (CPH2693) feat(device) detect new brand CPDEVICE: Apollo 10 Pro feat(device) detect brand OPPO: K12x 5G (CPH2667), A3x (CPH2641) feat(device) detect brand OnePlus: Nord 4 (CPH2661, CPH2663) feat(device) detect brand MobiWire: Smart P24 feat(device) detect brand Vivo: V17 Neo (1907_19) feat(device) detect brand Casper: VIA L40 feat(device) detect brand Huawei: MatePad SE 11 WiFi (AGS6-W09), MatePad SE 11 (AGS6-L09), MatePad T10s (AGS3-AL09) feat(device) detect brand Pico: A8110 feat(device) detect brand Xiaomi: Redmi Note 8 (2021) (biloba) feat(device) detect brand Blu: View 5 (B160V) feat(device) detect brand iHunt: Titan P10000 Pro feat(device) detect brand iTel: Vision 2 feat(device) detect new brand XPPen: Magic Drawing Pad feat(device) detect new brand Korax: Eurobox feat(device) detect brand Xiaomi: Redmi Note 7 (M1901F7G), Redmi 7A (M1903C3EG) feat(device) detect brand Vivo: Y85A, Y85, Y83A, Y71, Y71A feat(device) detect brand Maze Speed: M1582C Max feat(device) detect brand Dragon Touch: Y88X Plus feat(device) detect brand Vorcom: Quartz Pro feat(device) detect brand Lenovo: Vibe K5 (A6020l37), Vibe K5 Plus (A6020a46) feat(device) detect brand Huawei: Nova 5T (Yale-L71A, Yale-L61D), Honor Play 3, Honor 9X Pro, Honor 9X, Honor 8S, Honor 8A, Honor 20, Honor 10I feat(client) remove default engine from Wolvic browser feat(client) added multy browser family for Wolvic browser --- feat(sync) Improves detection for various bots (matomo-org#7857) * Improves detection for generic bots * Adds detection for Inspici * Adds detection for Meta-ExternalAgent * Adds detection for Meta-ExternalFetcher * Fix url for Facebook crawlers * chore: generate indexes
sanchezzzhak · Oct 7, 2024 · e6ba309 · e6ba309
1 parent 9697caf
commit e6ba309
Show file tree

Hide file tree

Showing 42 changed files with 3,169 additions and 481 deletions.
diff --git a/README.md b/README.md
diff --git a/client-hints.d.ts b/client-hints.d.ts
@@ -37,6 +37,7 @@ export interface ResultDevicePropClientHints {
 
 export interface ResultClientHints {
   upgradeHeader: boolean;
+  formFactors?: string[];
   meta?: ResultMetaClientHints;
   prefers?: ResultPrefersClientHints;
   os: ResultOsPropClientHints;

diff --git a/client-hints.js b/client-hints.js
@@ -46,6 +46,7 @@ const CH_UA_FORM_FACTORS = 'sec-ch-ua-form-factors';
   'sec-ch-ua-platform-version',
   'sec-ch-viewport-height',
   'sec-ch-viewport-width',
+  'sec-ch-ua-from-factors',
   'sec-ch-width',
   'ua',
   'ua-arch',
@@ -107,48 +108,48 @@ class ClientHints {
    * @param {ResultClientHints|JSONObject} result
    * @private
    */
-  __parseHints(hints, result) {
+  #parseHints(hints, result) {
     for (let key in hints) {
       let value = hints[key];
-      let lowerCaseKey = key.toLowerCase().replace('_', '-');
+      const lowerCaseKey = key.toLowerCase().replace('_', '-');
 
       switch (lowerCaseKey) {
         case 'http-sec-ch-ua-arch':
         case 'sec-ch-ua-arch':
         case 'arch':
         case 'architecture':
-          result.os.platform = helper.trimChars(value, '"');
+          result.os.platform = this.#trim(value);
           break;
         case 'http-sec-ch-ua-bitness':
         case 'sec-ch-ua-bitness':
         case 'bitness':
-          result.os.bitness = helper.trimChars(value, '"');
+          result.os.bitness = this.#trim(value);
           break;
         case 'http-sec-ch-ua-mobile':
         case 'sec-ch-ua-mobile':
         case 'mobile':
-          result.isMobile = true === value || '1' === value || '?1' === value;
+          result.isMobile = this.#bool(value);
           break;
         case 'http-sec-ch-ua-model':
         case 'sec-ch-ua-model':
         case 'model':
-          result.device.model = helper.trimChars(value, '"');
+          result.device.model = this.#trim(value);
           break;
         case 'http-sec-ch-ua-full-version':
         case 'sec-ch-ua-full-version':
         case 'uafullversion':
           result.upgradeHeader = true;
-          result.client.version = helper.trimChars(value, '"');
+          result.client.version = this.#trim(value);
           break;
         case 'http-sec-ch-ua-platform':
         case 'sec-ch-ua-platform':
         case 'platform':
-          result.os.name = helper.trimChars(value, '"');
+          result.os.name = this.#trim(value);
           break;
         case 'http-sec-ch-ua-platform-version':
         case 'sec-ch-ua-platform-version':
         case 'platformversion':
-          result.os.version = helper.trimChars(value, '"');
+          result.os.version = this.#trim(value);
           break;
         case 'brands':
           if (result.client.brands.length > 0) {
@@ -168,48 +169,88 @@ class ClientHints {
         // eslint-disable-next-line no-fallthrough
         case 'http-sec-ch-ua-full-version-list':
         case 'sec-ch-ua-full-version-list':
-          let pattern = new RegExp('"([^"]+)"; ?v="([^"]+)"(?:, )?', 'gi');
-          let items = [];
-          let matches = null;
-          while (matches = pattern.exec(value)) {
-            let brand = matches[1];
-            let skip = brand.indexOf('Not;A') !== -1 || brand.indexOf('Not A;') !== -1 || brand.indexOf('Not.A') !== -1;
-            if (skip) {
-              continue;
-            }
-            items.push({ brand, version: helper.trimChars(matches[2], '"') });
-          }
+          const items = this.#parseFullVersionList(value);
           if (items.length > 0) {
             result.client.brands = items;
           }
           break;
         case 'x-requested-with':
         case 'http-x-requested-with':
-          result.app = value;
-          if (value.toLowerCase() === 'xmlhttprequest') {
-            result.app = '';
-          }
+          result.app = this.#parseApp(value)
           break;
         case 'formfactors':
-          result.formFactors = value.map(val => val.toLowerCase());
-          break;
         case 'http-sec-ch-ua-form-factors':
         case 'sec-ch-ua-form-factors':
-          let matchFactors = /"([a-z]+)"/i.exec(value.toLowerCase());
-          if (matchFactors && matchFactors[1]) {
-            result.formFactors = matchFactors[1];
-          }
+          result.formFactors = this.#parseFormFactor(value);
           break;
       }
     }
   }
 
+  /**
+   * @param {boolean|string} value
+   * @return {boolean}
+   */
+  #bool(value) {
+    return true === value || '1' === value || '?1' === value;
+  }
+
+  /**
+   * @param {string} value
+   * @return {string}
+   */
+  #trim(value) {
+    return helper.trimChars(value, '"');
+  }
+
+  /**
+   * @param {string} value
+   * @return {string}
+   */
+  #parseApp(value) {
+    return value.toLowerCase() === 'xmlhttprequest' ? '' : value;
+  }
+
+  /**
+   * @param {string} value
+   * @return {[]}
+   */
+  #parseFullVersionList(value) {
+    const skipBrands = ['Not;A', 'Not A;', 'Not.A'];
+    const pattern = new RegExp('"([^"]+)"; ?v="([^"]+)"(?:, )?', 'gi');
+    const items = [];
+
+    let matches = null;
+    while ((matches = pattern.exec(value)) !== null) {
+      const brand = matches[1];
+      if (skipBrands.some(item => brand.includes(item))) {
+        continue;
+      }
+      items.push({ brand, version: helper.trimChars(matches[2], '"') });
+    }
+    return items;
+  }
+  /**
+   * @param {string|string[]} value
+   * @return {string[]}
+   */
+  #parseFormFactor(value) {
+    if (Array.isArray(value)) {
+      return value.map(val => val.toLowerCase());
+    }
+
+    const matches = value.toLowerCase().match(/"([a-z]+)"/gi);
+    return matches!== null ? matches.map(formFactor => {
+      return helper.trimChars(formFactor, '"')
+    }): [];
+  }
+
   /**
    * @param {JSONObject|{}} meta
    * @param {ResultClientHints} result
    * @private
    */
-  __parseMeta(meta, result) {
+  #parseMeta(meta, result) {
     for (let key in meta) {
       let value = meta[key];
       let lowerCaseKey = key.toLowerCase();
@@ -242,7 +283,7 @@ class ClientHints {
    * @return {ResultClientHints|JSONObject|{}}
    * @private
    */
-  __blank() {
+  #blank() {
     return {
       upgradeHeader: false,
       isMobile: false,
@@ -269,9 +310,9 @@ class ClientHints {
    * @return {ResultClientHints}
    */
   parse(hints, meta = {}) {
-    let result = this.__blank()
-    this.__parseHints(hints, result);
-    this.__parseMeta(meta, result);
+    let result = this.#blank()
+    this.#parseHints(hints, result);
+    this.#parseMeta(meta, result);
     return result;
   }
 

diff --git a/index.js b/index.js
@@ -1,5 +1,5 @@
 const helper = require('./parser/helper');
-const {attr} = helper;
+const { attr } = helper;
 
 // device parsers
 const MobileParser = require('./parser/device/mobile');
@@ -40,6 +40,7 @@ const APPLE_OS_LIST = require('./parser/const/apple-os');
 const DESKTOP_OS_LIST = require('./parser/const/desktop-os');
 const DEVICE_PARSER_LIST = require('./parser/const/device-parser');
 const CLIENT_PARSER_LIST = require('./parser/const/client-parser');
+const FORM_FACTORS_MAPPING = require('./parser/const/form-factor-mapping');
 const MOBILE_BROWSER_LIST = require('./parser/client/browser-short-mobile');
 const { hasUserAgentClientHintsFragment, hasDeviceModelByClientHints } = require('./parser/helper');
 const VENDOR_FRAGMENT_PARSER = 'VendorFragment';
@@ -490,6 +491,23 @@ class DeviceDetector {
     let clientFamily = attr(clientData, 'family', '');
     let deviceType = attr(deviceData, 'type', '');
 
+
+    // client hint detect device type
+    if (
+      deviceType === '' &&
+      clientHints &&
+      clientHints.device &&
+      clientHints.device.model &&
+      clientHints.formFactors.length
+    ) {
+      for(const [type, deviceTypeName] of Object.entries(FORM_FACTORS_MAPPING)) {
+          if (clientHints.formFactors.includes(type)) {
+            deviceType = '' + deviceTypeName;
+            break;
+          }
+      }
+    }
+
     /**
      * All devices containing VR fragment are assumed to be a wearable
      */
@@ -505,11 +523,9 @@ class DeviceDetector {
      *       a detected browser, but can still be detected. So we check the useragent for Chrome instead.
      */
     if (deviceType === '' && osFamily === 'Android' && helper.matchUserAgent('Chrome/[.0-9]*', userAgent)) {
-      if (helper.matchUserAgent('(Mobile|eliboM)', userAgent) !== null) {
-        deviceType = DEVICE_TYPE.SMARTPHONE;
-      } else{
-        deviceType = DEVICE_TYPE.TABLET;
-      }
+      deviceType = helper.matchUserAgent('(Mobile|eliboM)', userAgent)
+        ? DEVICE_TYPE.SMARTPHONE
+        : DEVICE_TYPE.TABLET;
     }
 
     /**
@@ -578,33 +594,32 @@ class DeviceDetector {
      */
     if (
       deviceType === '' &&
-      (osName === 'Windows RT' ||
-        (osName === 'Windows' && helper.versionCompare(osVersion, '8') >= 0)) &&
+      (osName === 'Windows RT' || (osName === 'Windows' && helper.versionCompare(osVersion, '8') >= 0)) &&
       helper.hasTouchFragment(userAgent)
     ) {
       deviceType = DEVICE_TYPE.TABLET;
     }
-    /**
-     * All devices running Puffin Secure Browser that contain letter 'D' are assumed to be desktops
-     */
-    if (deviceType === '' && helper.hasPuffinDesktopFragment(userAgent)) {
-      deviceType = DEVICE_TYPE.DESKTOP;
-    }
-
-    /**
-     * All devices running Puffin Web Browser that contain letter 'P' are assumed to be smartphones
-     */
-    if (deviceType === '' && helper.hasPuffinSmartphoneFragment(userAgent)) {
-      deviceType = DEVICE_TYPE.SMARTPHONE;
-    }
 
-    /**
-     * All devices running Puffin Web Browser that contain letter 'T' are assumed to be tablets
-     */
-    if (deviceType === '' && helper.hasPuffinTabletFragment(userAgent)) {
-      deviceType = DEVICE_TYPE.TABLET;
+    if (deviceType === '' && /Puffin\/\d/i.test(userAgent)) {
+      /**
+       * All devices running Puffin Secure Browser that contain letter 'D' are assumed to be desktops
+       */
+      if (helper.hasPuffinDesktopFragment(userAgent)) {
+        deviceType = DEVICE_TYPE.DESKTOP;
+      }
+      /**
+       * All devices running Puffin Web Browser that contain letter 'P' are assumed to be smartphones
+       */
+      if (helper.hasPuffinSmartphoneFragment(userAgent)) {
+        deviceType = DEVICE_TYPE.SMARTPHONE;
+      }
+      /**
+       * All devices running Puffin Web Browser that contain letter 'T' are assumed to be tablets
+       */
+      if (helper.hasPuffinTabletFragment(userAgent)) {
+        deviceType = DEVICE_TYPE.TABLET;
+      }
     }
-
     /**
      * All devices running Opera TV Store are assumed to be a tv
      */
@@ -699,8 +714,8 @@ class DeviceDetector {
     if (deviceModel !== '' && helper.hasUserAgentClientHintsFragment(userAgent)) {
       const osHints = attr(clientHints, 'os', {});
       const osVersion = attr(osHints, 'version', '');
-      return userAgent.replace(/(Android 10[.\d]*; K)/,
-        `Android ${osVersion !== '' ? osVersion: '10'}; ${deviceModel}`
+      return userAgent.replace(/(Android (?:10[.\d]*; K|1[1-5]))/,
+        `Android ${osVersion !== '' ? osVersion : '10'}; ${deviceModel}`
       );
     }
 
@@ -733,7 +748,8 @@ class DeviceDetector {
       trusted: null
     };
 
-    if (!helper.hasDeviceModelByClientHints(clientHints) && helper.hasUserAgentClientHintsFragment(ua)) {
+    // skip all parse is client-hints useragent and model not exist
+    if (!helper.hasDeviceModelByClientHints(clientHints) && helper.hasUserAgentClientHintsFragment(userAgent)) {
       return Object.assign({}, result);
     }
 
@@ -769,7 +785,7 @@ class DeviceDetector {
       }
     }
 
-    // client hints
+    // client hints get model raw
     if (result.model === '' && helper.hasDeviceModelByClientHints(clientHints)) {
       result.model = clientHints.device.model;
     }

diff --git a/parser/client/browser-families.js b/parser/client/browser-families.js
@@ -36,14 +36,14 @@ module.exports = {
   'K4', 'WK', 'T3', 'K5', 'MU', '9P', 'K6', 'VR', 'N9',
   'M9', 'F9', '0P', '0A', 'JR', 'D3', 'TK', 'BP', '2F',
   '2M', 'K7', '1N', '8A', 'H7', 'X3', 'T4', 'X4', '5O',
-  '8C', '3M', '6I', '2P', 'PU', '7I', 'X5',
+  '8C', '3M', '6I', '2P', 'PU', '7I', 'X5', 'AL',
   ],
   'Firefox': [
   'FF', 'BI', 'BF', 'BH', 'BN', 'C0', 'CU', 'EI', 'F1',
   'FB', 'FE', 'AX', 'FM', 'FR', 'FY', 'GZ', 'I4', 'IF',
   'IW', 'LH', 'LY', 'MB', 'MN', 'MO', 'MY', 'OA', 'OS',
   'PI', 'PX', 'QA', 'S5', 'SX', 'TF', 'TO', 'WF', 'ZV',
-  'FP', 'AD', 'WL', '2I', 'P9', 'KJ', 'WY', 'VK', 'W5',
+  'FP', 'AD', '2I', 'P9', 'KJ', 'WY', 'VK', 'W5',
   '7C', 'N7', 'W7', '8P',
   ],
   'Internet Explorer': ['IE', 'CZ', 'BZ', 'IM', 'PS', '3A', '4A', 'RN'],

diff --git a/parser/client/browser.js b/parser/client/browser.js
@@ -189,6 +189,15 @@ class Browser extends ClientAbstractParser {
       engineVersion = '';
     }
 
+    if ('Wolvic' === name) {
+      if ('Blink' === engine) {
+        family = 'Chrome';
+      }
+      if ('Gecko' === engine) {
+        family = 'Firefox';
+      }
+    }
+
     if (name === '') {
       return null;
     }

diff --git a/parser/const/form-factor-mapping.js b/parser/const/form-factor-mapping.js
@@ -0,0 +1,11 @@
+const DEVICE_TYPE = require('./device-type');
+
+module.exports =  {
+  'automotive' : DEVICE_TYPE.CAR_BROWSER,
+  'xr'         : DEVICE_TYPE.WEARABLE,
+  'watch'      : DEVICE_TYPE.WEARABLE,
+  'mobile'     : DEVICE_TYPE.SMARTPHONE,
+  'tablet'     : DEVICE_TYPE.TABLET,
+  'desktop'    : DEVICE_TYPE.DESKTOP,
+  'eink'       : DEVICE_TYPE.TABLET,
+};