warn: [Entity] Unknown entity type

[danizwam]

Hi,

today i got these errors while runnig the script.

2015-01-13T19:24:23.094Z - warn: [Entity] Unknown entity type, id=xxxxxxxxxxxxxxxxxxxx, type=undefined

Entity.coffee contains the following code.

        if data.type is 'portal'
            createPortalEntity id, timestamp, data, callback
        else if data.type is 'region'
            createFieldEntity id, timestamp, data, callback
        else if data.type is 'edge'
            createLinkEntity id, timestamp, data, callback
        else
            logger.warn "[Entity] Unknown entity type, id=#{id}, type=#{data.type}"
            callback && callback()

It seems it is the same prob iitc got three days ago:

10th January 2015

IITC 0.20.0 released. This is a critical update to fix loading of portals/links/fields after a protocol change by Niantic.

http://iitc.jonatkins.com/

Does anybody know how to fix this?

Edit:

I played a little bit with the js-Files and that is what i found:

Niantic changed the format of the "data".

request-portals.js

About line 282 i created some log entries:

      if (t.tile.gameEntities != null) {
        return async.each(t.tile.gameEntities, function(entity, callback) {
          return Entity.add(entity[0], entity[1], entity[2], function(type) {
          logger.info("####: ID=" + entity[0]);
          logger.info("####: TIMESTAMP=" + entity[1]);
          logger.info("####: DATA=" + entity[2]);

Portal.

2015-01-13T20:22:55.743Z - info: ####: ID=58701dcc7c64422ca0b5ccbd2fc8fffd.16
2015-01-13T20:22:55.743Z - info: ####: TIMESTAMP=1417021375226
2015-01-13T20:22:55.743Z - info: ####: DATA=p,N,52566862,7593227,1,0,0,http://lh5.ggpht.com/bgyvgnnNqjflA6fgH_7TfJ_dxeDeoDS12zaL9L3azhFK9Qt67kOyAqQZklxNpOk7BHGsZeYdqBSUm-fWmP4tZw,Luna Park,

You can see, the "DATA" is now an array:

TYPE,FACTION,LAT,LNG,LEVEL,XM,RESONATOR-COUNT,PICTURE-URL,TITLE

Edge:

2015-01-13T20:26:05.741Z - info: ####: ID=647b7ba8c8224e88b835f5b26b78ebfd.9
2015-01-13T20:26:05.741Z - info: ####: TIMESTAMP=1421093566634
2015-01-13T20:26:05.741Z - info: ####: DATA=e,E,1025d2fa138143dcb61e8dcd87b46c2a.16,52716320,6767534,7e407a37512348be8f747429289810c5.16,52833075,6366633

Field:

2015-01-13T20:28:25.307Z - info: ####: ID=5e33289ce78a4d6c9b4c7de4f62ea931.b
2015-01-13T20:28:25.307Z - info: ####: TIMESTAMP=1420818476250
2015-01-13T20:28:25.307Z - info: ####: DATA=r,R,618813f6d82c4c6d810c729ddf6a6dcd.16,52527632,7196913,bc00081f51594dd8b8c4f432e43bb111.16,52443113,7091642,7a963deaf168482f884af9390f596658.16,52691670,7270650

entity.js:

I changed lines like data.type == 'portal' to data[0] === 'p'

add: function(id, timestamp, data, callback) {
  var main;
  main = function() {
    Entity.entityCount++;
    if (data[0] === 'p') {
      return createPortalEntity(id, timestamp, data, callback);
    } 
    else if (data[0] === 'r') {
      return createFieldEntity(id, timestamp, data, callback);
    } else if (data[0] === 'e') {
      return createLinkEntity(id, timestamp, data, callback);
    } else {
      logger.warn("[Entity] Unknown entity type, id=" + id + ", type=" + data[0]);
      return callback && callback();
   }
  };

---

In createPortalEntity i changed the calculation of lat lng and the if statement.

createPortalEntity = function(id, timestamp, data, callback) {
data[2] = data[2] / 1e6;
data[3] = data[3] / 1e6;
return createEntity('Portals', id, timestamp, data, function() {
if (data[1] !== 'N' && argv.detail !== 'false') {
return Entity.requestPortalDetail(id, function() {
return callback && callback('portal');
});
} else {
return callback && callback('portal');
}
});
};

Afterwards, the scripts calls the createEntity method:

createEntity = function(collection, id, timestamp, data, callback) {
data.time = timestamp;
return Database.db.collection(collection).update({
_id: id
}, {
$set: data
}, {
upsert: true
}, function(err) {
if (err) {
logger.error('[Entity] Failed to insert entity (id=%s) into database: %s', id, err.message);
}
return callback();
});
};

This causes this kind of error:

2015-01-13T20:28:25.300Z - error: [Entity] Failed to insert entity (id=8d20d05d599d4207af7cd41cae7f2391.9) into database: Invalid modifier specified: $set

I think "data" was in json and Database.db.collection could handle this, but it isn't anymore :(

I'm stuck and i don't know what to to next :(

Perhaps somebody can use my "work" ;) to fix this issue!

[danieldubrovski]

that is what i added into entity.coffee (only for portals)
the idea is just to remap into old objects. and that's it

diff --git a/src/lib/entity.coffee b/src/lib/entity.coffee
index 005f472..edfa4d3 100644
--- a/src/lib/entity.coffee
+++ b/src/lib/entity.coffee
@@ -10,11 +10,11 @@ Entity = GLOBAL.Entity =
         portals: 0
         fields:  0
         links:   0

     entityCount: 0

     add: (id, timestamp, data, callback) ->
+        data = remap data
         # update counter every 100 entities

         main = ->
@@ -24,9 +24,11 @@ Entity = GLOBAL.Entity =
             if data.type is 'portal'
                 createPortalEntity id, timestamp, data, callback
             else if data.type is 'region'
-                createFieldEntity id, timestamp, data, callback
+                return callback();
+                #createFieldEntity id, timestamp, data, callback
             else if data.type is 'edge'
-                createLinkEntity id, timestamp, data, callback
+                return callback();
+                #createLinkEntity id, timestamp, data, callback
             else
                 logger.warn "[Entity] Unknown entity type, id=#{id}, type=#{data.type}"
                 callback && callback()
@@ -80,27 +82,39 @@ Entity = GLOBAL.Entity =
                 callback()

             onSuccess: (response, callback) ->
+                response = remap response.result

                 if response.capturedTime?
                     response.capturedTime = parseInt response.capturedTime

@@ -182,3 +198,60 @@ createLinkEntity = (id, timestamp, data, callback) ->

     createEntity 'Links', id, timestamp, data, ->
         callback && callback 'link'
+        
+types = {
+    'p' : 'portal'
+    'r' : 'region'
+    'e' : 'edge'
+}
+        
+teams = {
+    'E' : 'ENLIGHTENED'
+    'R' : 'RESISTANCE'
+    'N' : 'NEUTRAL'
+}
+    
+remap = (data) ->
+#TODO: links and fields
+    result = {
+        type: types[data[0]]
+        team: teams[data[1]]
+        latE6: data[2]
+        lngE6: data[3]
+        level: data[4]
+        health: data[5]
+        resCount: data[6]
+        image: data[7]
+        title: data[8]
+        ornaments: data[9]
+    }
+    result.mods = remapMods(data[10]) if data[10]
+    result.resonators = remapResos(data[11]) if data[11]
+    result.owner = data[12] if data[12]
+    return result
+     
+
+remapMods = (mods) ->
+    result = []
+    for mod in mods
+        result.push(if mod then {
+            owner: mod[0]
+            name: mod[1]
+            rarity: mod[2]
+            stats: mod[3]
+        } else null);
+    return result;
+    
+remapResos = (resos) ->
+    result = []
+    for reso in resos
+        result.push(if reso then {
+            owner: reso[0]
+            level: reso[1]
+            energy: reso[2]
+        } else null);
+    return result;
+    
+    
+    
+    

[danizwam]

I had the same idea, but i didn't know the correct format and how to do this ;)

Nice work btw :D

I'll give it a try when i'm at home

[danizwam]

I have trouble merging your changes without a tool ;) I got an error with "main ->". Could you please post your new entity.coffee?

Running "coffee:project" (coffee) task

src/lib/entity.coffee:20:1: error: unexpected indentation
main = ->
^^^^^^^^
In file: src/lib/entity.coffee
On line: 19
main = ->
^
Warning: CoffeeScript failed to compile. Use --force to continue.

[danieldubrovski]

my entity.coffee has a lot of other changes.
but as error says, just check your indentation according to coffeescript idea

[danizwam]

I'm not that much into coffee.. I'm more into java and javascript. So i compiled the entitiy.js without remapping and added data = remap(data); manually.

add: function(id, timestamp, data, callback) {
var main;
data = remap(data);
main = function() {
Entity.entityCount++;
if (data.type === 'portal') {
return createPortalEntity(id, timestamp, data, callback);
} else if (data.type === 'region') {
return createFieldEntity(id, timestamp, data, callback);
} else if (data.type === 'edge') {
return createLinkEntity(id, timestamp, data, callback);
} else {
logger.warn("[Entity] Unknown entity type, id=" + id + ", type=" + data.type);
return callback && callback();
}
};

I don't know why, but adding "data = remap data" to the entitiy.coffee always causes a compile error.
The script runs now flawless and my database has been updated ;)

If you have a lot of more changes in your entity.coffee, does that mean you're maintaining the project now?

[danieldubrovski]

no, more like personal additions.
i also suggest we remove our spam comments

P.S. dont forget to remap details response also

[danizwam]

What do you mean?

I've added every change suggested by you without "data = remap data" and compiled the entity.js. Then i added "data = rempa(data);" to the entity.js.
Did i miss something?

[danieldubrovski]

in order for details scan to work

@@ -80,27 +82,39 @@ Entity = GLOBAL.Entity =
                 callback()

             onSuccess: (response, callback) ->
+                response = remap response.result

                 if response.capturedTime?
                     response.capturedTime = parseInt response.capturedTime

[sn0opy]

The whole file with the applied diff: https://gist.github.com/sn0opy/37f0b1fe40f5cf6f5910

[sasha2002]

@sn0opy can you help me with this ?
https://github.com/breeswish/sh-enl-ingress-intel-chat-bot
I integrate ingress-exporter that work very well as single app, but when i try to make a diff mungedetector.coffee and another files, i got for now this :

node build/app.js --debug true

2015-01-27T18:37:36.710Z - info: [argv] --debug true

/somedir/build/lib/requestfactory.js:13
delayedRequestQueue = new delayedQueue(function(task) {
^
TypeError: object is not a function
at Object. (/somedir/build/lib/requestfactory.js:13:25)
at Object. (/somedir/build/lib/requestfactory.js:291:4)
at Module._compile (module.js:456:26)
at Object.Module._extensions..js (module.js:474:10)
at Module.load (module.js:356:32)
at Function.Module._load (module.js:312:12)
at Module.require (module.js:364:17)
at require (module.js:380:17)
at Object. (/somedir/build/lib/mungedetector.js:6:20)
at Object. (/somedir/build/lib/mungedetector.js:216:4)