[ContourClock] suboptimal rendering - font characters in json files are not pretokenized #3649
Comments
|
Wow, that sucks. Surely this should just happen when the clock starts up? That would be an easy fix. The JSON files won't be pretokenised or anything, so the simplest 'fix' would be to make them JS modules instead, and to not do |
|
I just tried a new approach. Font files are parsed when you select a font, then the images are stored in the settings file. Rendering time is now 250ms instead of 50ms, but RAM usage is now 4680 all the time, because the settings object stays in RAM. I wonder which way i should optimize this - RAM usage, storage or performance. |
|
... you could try what I suggested with JS modules and get low ram usage and pretty good performance? Storing the contents in settings JSON feels like it's not really too different from what was there before. It's still a JSON object that's parsed - it's just whether you decide to parse it once when the app loads, or every time you draw something. |
So it is just about adding atob inside the json file, renaming it from .json to .js and loading it here via but still a bit better would be to load the font variable once somewhere in app.js instead of the drawClock |
If you're using If you want to use module then you need |
|
Anyway, the minimalistic improvement is to cache the font by fontIndex passed to drawclock here var is12;
var font;
var index;
function getHours(d) {
var h = d.getHours();
if (is12===undefined) is12 = (require('Storage').readJSON('setting.json',1)||{})["12hour"];
if (!is12) return h;
return (h%12==0) ? 12 : h%12;
}
exports.drawClock = function(fontIndex) {
var digits = [];
if (fontIndex!=index) {
var fontFile=require("Storage").read("contourclock-"+Math.abs(parseInt(fontIndex+0.5))+".json");
if (fontFile==undefined) return(false); //exit if font file not found
font = JSON.parse(fontFile);
index=fontIndex;
}This way it is parsed only once as long as the index passed to drawClock is same. So it is only slow at app startup but does not consume cpu by reloading font at every draw.
But then we are back to first question, when will apploader optimize the file? it is only about filename change json->js? or is it only done on files in metadata here https://github.com/espruino/BangleApps/blob/master/apps/contourclock/metadata.json#L13 ? Fonts are uploaded here https://github.com/espruino/BangleApps/blob/master/apps/contourclock/custom.html#L59 BTW maybe all fonts could be uploaded and custom html could be removed, maybe it is not so much data after all ? EDIT: but this belongs to issue #3648 |
|
Parsing the font file takes about 200ms, which is acceptable on startup in my opinion.
One font takes up about 20kB, uploading all fonts uses a significant portion of the available flash. It would also slow down updating unnecessarily. |
yes, maybe it is too much, but if you would store it with atob then it would be 2/3 of that and with heatshrink applied it would be even (much?) smaller. and maybe decompressing heatshrink would not be slower than current atob |
oh, sorry, when looking at the code, better would be to cache the digits array instead, which comes in next lines here https://github.com/espruino/BangleApps/blob/master/apps/contourclock/lib.js#L14 this works for me diff --git a/apps/contourclock/lib.js b/apps/contourclock/lib.js
index c4f927953..161c313b1 100644
--- a/apps/contourclock/lib.js
+++ b/apps/contourclock/lib.js
@@ -1,4 +1,24 @@
var is12;
+var digits;
+var index;
+var fontName;
+function cacheDigits(fontIndex){
+ let fontFile=require("Storage").read("contourclock-"+Math.abs(parseInt(fontIndex+0.5))+".json");
+ if (fontFile==undefined) return; //exit if font file not found
+ let font = JSON.parse(fontFile);
+ digits=[];
+ for (var n in font.characters) {
+ digits.push({width: parseInt(font.characters[n].width),
+ height: font.size,
+ bpp: 2,
+ transparent: 1,
+ buffer:E.toArrayBuffer(atob(font.characters[n].buffer))});
+ }
+ if (n!=10) return; //font file seems to be invalid
+ fontName=font.name;
+ index=fontIndex;
+}
+
function getHours(d) {
var h = d.getHours();
if (is12===undefined) is12 = (require('Storage').readJSON('setting.json',1)||{})["12hour"];
@@ -7,18 +27,10 @@ function getHours(d) {
}
exports.drawClock = function(fontIndex) {
- var digits = [];
- fontFile=require("Storage").read("contourclock-"+Math.abs(parseInt(fontIndex+0.5))+".json");
- if (fontFile==undefined) return(false); //exit if font file not found
- var font = JSON.parse(fontFile);
- for (var n in font.characters) {
- digits.push({width: parseInt(font.characters[n].width),
- height: font.size,
- bpp: 2,
- transparent: 1,
- buffer:E.toArrayBuffer(atob(font.characters[n].buffer))});
- }
- if (n!=10) return (false); //font file seems to be invalid
+ if (fontIndex!=index) {
+ cacheDigits(fontIndex);
+ if (fontIndex!=index) return false;
+ }
var x=0;
var y = g.getHeight()/2-digits[0].height/2;
var date = new Date();
@@ -53,5 +65,5 @@ exports.drawClock = function(fontIndex) {
g.setColor(fg);
g.setBgColor(bg);
}
- return font.name;
+ return fontName;
}EDIT: and also BTW all those d1-d5,w1-w5 variables could be defined with let/var as local variables inside drawClock, now they are global variables (just connect to watch and type its name to see) |
just tested it with making font into module and also tested heatshrink on thedata anton.json is just uploaded font-Anton.json from ide as is with tokenization it can be seen that the pretokenization saves space by tokenizing atob into binary strings. so even in current state you could compress the arrays and then when loading digits decompress it back. that would save quite some storage space and with recent changes would be done only once. |
|
I have implemented the module way and converted only one font to this mode (=loaded "contourclock-3.json" into IDE from storage, added function cacheDigits(fontIndex){
var font;
let fn = "contourclock-"+Math.abs(parseInt(fontIndex+0.5));
try {
// new way
font=require(fn+".js");
digits=[];
for (var n in font.characters) {
digits.push({width: parseInt(font.characters[n].width),
height: font.size,
bpp: 2,
transparent: 1,
buffer:font.characters[n].buffer});
}
} catch {
// fallback to old way
let fontFile=require("Storage").read(fn+".json");
if (fontFile==undefined) return; //exit if font file not found
font = JSON.parse(fontFile);
digits=[];
for (var n in font.characters) {
digits.push({width: parseInt(font.characters[n].width),
height: font.size,
bpp: 2,
transparent: 1,
buffer:E.toArrayBuffer(atob(font.characters[n].buffer))});
}
}
if (n!=10) return; //font file seems to be invalid
fontName=font.name;
index=fontIndex;
}and when you go to preferences and slide between fonts is is very noticeable that the one converted is much faster to show |
|
Have you seen https://www.espruino.com/Bangle.js+Performance#measuring-performance - you might be able to get some numbers |
|
Thanks for those tips. I have just added I have also made font 1 = "Oswald" into module and compressed the array via heatshrink. font 3 = "Beba Neue" is only module with no compression. Here is output of sliding in preferences few times to right, left, right. then keeping "Bebas Neue" and letting the clock run for few minutes So it depends on font images a bit but Font sizes are my current function cacheDigits(fontIndex){
var font;
let fn = "contourclock-"+Math.abs(parseInt(fontIndex+0.5));
var n;
try {
font=require(fn+".js");
digits=[];
n=font.characters.length-1;
let getBuff=function(c){if (c.shrink) return require("heatshrink").decompress(c.shrink); else return c.buffer;}
font.characters.forEach((c)=> {
digits.push({width: parseInt(c.width),
height: font.size,
bpp: 2,
transparent: 1,
buffer:getBuff(c)});
})
} catch {
let fontFile=require("Storage").read(fn+".json");
if (fontFile==undefined) return; //exit if font file not found
font = JSON.parse(fontFile);
digits=[];
for (n in font.characters) {
digits.push({width: parseInt(font.characters[n].width),
height: font.size,
bpp: 2,
transparent: 1,
buffer:E.toArrayBuffer(atob(font.characters[n].buffer))});
}
}
if (n!=10) return; //font file seems to be invalid
fontName=font.name;
index=fontIndex;
}I have compressed font 1 like this then loaded json into webide and replaced |
BTW, I was using module because I did not know how to do it with eval. The IDE complains at upload if it the file is just object in {} with methods in JSON. However now after everything done I had an idea to wrap it into () and that works. So which is better / what is nicer about eval ? |
Ahh - I forgot about that. But I just had a thought! Actually And you could use Espruino to pretokenise the JSON and write it back to the file - it's a bit of a hack though: And it doesn't help with decoding the base64 either. So I'd say maybe just rename "contourclock-0.json" to "contourclock-0.js" and wrap the JS in Personally I feel like 20k for a JSON file isn't much - I'd just store the binary data the "contourclock-0.js" and not bother trying to decompress it on the fly (because if you do that the data ends up in RAM again) |
you don't need to but it needs to be inside the storage file, then also the IDE does not complain at upload EDIT: I mean you don't need to do |
OTOH when you just have the clock running most of the day drawing the data every minute all the day, it may not be bad for the five |
true, yes. And we're on Bangle.js 2 only with this app so I guess we do have the RAM available, so that would be fine |
Affected hardware version
Bangle 2
Your firmware version
2v24
The bug
All the digits are stored as base64 strings, look e.g. at https://github.com/espruino/BangleApps/blob/master/apps/contourclock/fonts/font-Anton.json
and then in drawClock() called each minute (?) the file is parsed and decoded here https://github.com/espruino/BangleApps/blob/master/apps/contourclock/lib.js#L19 over and over again.
Either the selected font file could be parsed once in the code when font selection is made and stored in new file (contourclock-font,json ?) with the array somehow stored as binary or maybe the font json files could contain atob() inside so that it gets pretokenized by BangleApps uploader automagically(?) This is mainly question to @gfwilliams - would such json with atob() inside be pretokenized at upload time and if not, would JSON.parse handle it at parse time as valid json? If not how to solve it - maybe eval on the json would work then?
Installed apps
No response
The text was updated successfully, but these errors were encountered: