not getting bulk.json

[00aixxia00]

Hi,
nice project !

but..
when i try running push_to_server.py I get this message:

bulk0.json
{"error":{"root_cause":[{"type":"parse_exception","reason":"request body is required"}],"type":"parse_exception","reason":"request body is required"},"status":400}

and if I look at my bulk.json it is empty. can you help me?

[DanteNiewenhuis]

I guess that the folder where the json files are supposed to be is empty.
This means that it will create bulk0.json but will never write anything into it.
This problem is either cause by the folder name on your pc not being the same as the folder name in bulk.py or the problem is caused by parse_to_json.py.
Do you have any json files in the dataset folder?

[00aixxia00]

Yes it was.
Just found the problem, I forgot to install lxml. Now there is some Json data inside, but I am getting:
bulk0.json
{"error":"Content-Type header [application/x-www-form-urlencoded] is not supported","status":406}

[lucasfijen]

Hi 00aixxia00,

It's been a while since we worked on this assignment.

I believe the latter error might be due to a different version of elastic. At the time we did this assignment we used Elastic Search 5.6.

According to this thread: https://stackoverflow.com/questions/47544966/elasticsearch-content-type-header-application-x-www-form-urlencoded-is-not-s it has to do with Elastic Search 6.0.

Do you happen to be using this new version?
Hope you can get the project working!

Edit:

So according to that thread, you could replace line 56 of push_to_server.py into:
os.system("curl -s -XPOST http://localhost:9200/_bulk -H 'Content-Type: application/json' --data-binary @json_bulks/" + bulk )

Let met know if this solves your problem!

[00aixxia00]

Yes this works now, thank you :)

still have some problems:
when im searching something i dont get a result but this message:

elasticsearch.exceptions.RequestError: RequestError(400, 'search_phase_execution_exception', 'field value function must not produce negative scores, but got: [-0.30000000447034836] for field value: [-3.0]')

and

ERROR in app: Exception on /posthandler [POST]

[lucasfijen]

Sounds like there is a negative float somewhere.
Could you print the input_dict that is the input variable of the function perform_search_from_dict in queryhandler.py?
If you add a print statement at the beginning of the function, it should print the dictionary that is sent towards elastic in the debugger of flask.

[00aixxia00]

{'TEXT': 'alcohol', 'CATEGORY': ['lifehacks'], 'EXCLUDE': [], 'DATE': None, 'SIZE': 10, 'SHOWCLOUD': False, 'DATETYPE': None}
[2019-05-28 17:58:36,515] ERROR in app: Exception on /posthandler [POST]
...
elasticsearch.exceptions.RequestError: RequestError(400, 'search_phase_execution_exception', 'field value function must not produce negative scores, but got: [-0.4000000059604645] for field value: [-4.0]')

and if itype in alc i am getting:

{'TEXT': 'alc', 'CATEGORY': ['lifehacks'], 'EXCLUDE': [], 'DATE': None, 'SIZE': 10, 'SHOWCLOUD': False, 'DATETYPE': None}
[2019-05-28 17:58:12,363] ERROR in app: Exception on /posthandler [POST]
...
File "Search_it_like_its_hot-master/html/queryhandler.py", line 10, in query_handler
if results['hits']['total'] > 0:
TypeError: '>' not supported between instances of 'dict' and 'int'

[lucasfijen]

You can try printing results['hits']['total'] as it seems that the value there is a dict instead of an int. Probably that is also version related. Perhaps you need further unpacking there.