Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

MapBox VectorTile: bug fixes using an official MapBox stream #2469

Open
wants to merge 19 commits into
base: master
Choose a base branch
from
+273 −110
Open

MapBox VectorTile: bug fixes using an official MapBox stream #2469

Show file tree
Hide file tree
Changes from 13 commits
Commits
Show all changes
19 commits
Select commit Hold shift + click to select a range
06804f1
feat(VectorTile): add support for relative url in style
ftoromanoff Oct 29, 2024
5c62908
fix(VectorTile): fix {z}/{y}/{x}
ftoromanoff Oct 29, 2024
f903a1a
fix(MVTStyle): icon properties -> fix return of function when id incl…
ftoromanoff Nov 25, 2024
328cc2c
feat(Source): add propertie 'warn' for avoiding multiple console.warn
ftoromanoff Oct 29, 2024
a96f21b
test(VectorTileSource): fix test
ftoromanoff Oct 29, 2024
d756acb
refactor(VectorTileParser): cleanup
ftoromanoff Nov 4, 2024
a881fac
fix(MVTLayers): add MVTLayer where MVTStyle.layer has 'ref' properties
ftoromanoff Nov 22, 2024
b77965b
fix(Style): take style.zoom into account for LabelLayer and Feature2T…
ftoromanoff Nov 6, 2024
7f27ef8
fix(examples): fix linked with zoom properties well used
ftoromanoff Nov 21, 2024
c7a1c32
fix(MVTParser): supp use of layer.style.zoom in parser
ftoromanoff Nov 25, 2024
6d568fa
fix(Style): Don't draw Polygon when fill.color is undefined
ftoromanoff Nov 22, 2024
5f63ee2
fix(Style): Don't draw stroke when width is 0
ftoromanoff Nov 26, 2024
af2e0f1
fix(LabelLayer): gestion simplified of line and polygon Label
ftoromanoff Nov 25, 2024
049dd6a
fix(MVTStyle): Doing recoloring only with sdf icons.
ftoromanoff Nov 21, 2024
efa1777
fix(Label): Multiple labels with same textContent
ftoromanoff Nov 26, 2024
36bcd33
refactor(MVTParser): 1 feature per vtfeature
ftoromanoff Nov 26, 2024
151e9af
fix(VectorTile): supp order in Style as it's only a Label properties …
ftoromanoff Nov 14, 2024
bb8f57d
example(MVT): add example with official MapBox style file
ftoromanoff Nov 26, 2024
18b1d2c
fix(Style): dont draw icon when size is 0
ftoromanoff Nov 27, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 0 additions & 1 deletion examples/source_file_kml_raster_usgs.html
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -69,7 +69,6 @@
}

var kmlStyle = {
zoom: { min: 10, max: 20 },
text: {
field: '{name}',
haloColor: 'black',
Expand Down
3 changes: 3 additions & 0 deletions src/Converter/Feature2Texture.js
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -70,6 +70,9 @@ function drawFeature(ctx, feature, extent, invCtxScale) {
for (const geometry of feature.geometries) {
if (Extent.intersectsExtent(geometry.extent, extent)) {
context.setGeometry(geometry);
if (style.zoom.min > style.context.zoom || style.zoom.max <= style.context.zoom) {
return;
}

if (
feature.type === FEATURE_TYPES.POINT && style.point
Expand Down
83 changes: 62 additions & 21 deletions src/Core/Style.js
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -87,14 +87,18 @@ async function loadImage(source) {
return (await promise).image;
}

function cropImage(img, cropValues = { width: img.naturalWidth, height: img.naturalHeight }) {
canvas.width = cropValues.width;
canvas.height = cropValues.height;
function cropImage(img, cropValues) {
const x = cropValues.x || 0;
const y = cropValues.y || 0;
const width = cropValues.width || img.naturalWidth;
const height = cropValues.height || img.naturalHeight;
canvas.width = width;
canvas.height = height;
const ctx = canvas.getContext('2d', { willReadFrequently: true });
ctx.drawImage(img,
cropValues.x || 0, cropValues.y || 0, cropValues.width, cropValues.height,
0, 0, cropValues.width, cropValues.height);
return ctx.getImageData(0, 0, cropValues.width, cropValues.height);
x, y, width, height,
0, 0, width, height);
return ctx.getImageData(0, 0, width, height);
}

function replaceWhitePxl(imgd, color, id) {
Expand Down Expand Up @@ -158,7 +162,7 @@ function defineStyleProperty(style, category, parameter, userValue, defaultValue
const dataValue = style.context.featureStyle?.[category]?.[parameter];
if (dataValue != undefined) { return readExpression(dataValue, style.context); }
if (defaultValue instanceof Function) {
return defaultValue(style.context.properties, style.context);
return defaultValue(style.context.properties, style.context) ?? defaultValue;
}
return defaultValue;
},
Expand Down Expand Up @@ -765,10 +769,11 @@ class Style {
* @param {Object} sprites vector tile layer.
* @param {Number} [order=0]
* @param {Boolean} [symbolToCircle=false]
* @param {Set} warn Set storing all warnings encountered by the source.
*
* @returns {StyleOptions} containing all properties for itowns.Style
*/
static setFromVectorTileLayer(layer, sprites, order = 0, symbolToCircle = false) {
static setFromVectorTileLayer(layer, sprites, order = 0, symbolToCircle = false, warn) {
const style = {
fill: {},
stroke: {},
Expand Down Expand Up @@ -804,7 +809,8 @@ class Style {
style.stroke.color = color;
style.stroke.opacity = opacity;
style.stroke.width = 1.0;
style.stroke.dasharray = [];
} else {
style.stroke.width = 0.0;
}
} else if (layer.type === 'line') {
const prepare = readVectorProperty(layer.paint['line-color'], { type: 'color' });
Expand All @@ -820,6 +826,8 @@ class Style {
style.point.opacity = opacity;
style.point.radius = readVectorProperty(layer.paint['circle-radius']);
} else if (layer.type === 'symbol') {
// if symbol we shouldn't draw stroke but defaut value is 1.
style.stroke.width = 0.0;
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

no need for this comment I think, the code is clear enough

// overlapping order
style.text.zOrder = readVectorProperty(layer.layout['symbol-z-order']);
if (style.text.zOrder == 'auto') {
Expand Down Expand Up @@ -861,6 +869,12 @@ class Style {
// additional icon
const iconImg = readVectorProperty(layer.layout['icon-image']);
if (iconImg) {
const cropValueDefault = {
x: 0,
y: 0,
width: 1,
height: 1,
};
try {
style.icon.id = iconImg;
if (iconImg.stops) {
Expand All @@ -871,22 +885,50 @@ class Style {
if (stop[1].includes('{')) {
cropValues = function _(p) {
const id = stop[1].replace(/\{(.+?)\}/g, (a, b) => (p[b] || '')).trim();
cropValues = sprites[id];
if (cropValues === undefined) {
const warning = `WARNING: "${id}" not found in sprite file`;
if (!warn.has(warning)) {
warn.add(warning);
console.warn(warning);
}
sprites[id] = cropValueDefault;// or return cropValueDefault;
}
return sprites[id];
};
} else if (cropValues === undefined) {
const warning = `WARNING: "${stop[1]}" not found in sprite file`;
if (!warn.has(warning)) {
warn.add(warning);
console.warn(warning);
}
cropValues = cropValueDefault;
}
return [stop[0], cropValues];
}),
};
style.icon.cropValues = iconCropValue;
} else {
style.icon.cropValues = sprites[iconImg];
if (iconImg[0].includes('{')) {
if (iconImg.includes('{')) {
style.icon.cropValues = function _(p) {
const id = iconImg.replace(/\{(.+?)\}/g, (a, b) => (p[b] || '')).trim();
style.icon.cropValues = sprites[id];
if (sprites[id] === undefined) {
const warning = `WARNING: "${id}" not found in sprite file`;
if (!warn.has(warning)) {
warn.add(warning);
console.warn(warning);
}
sprites[id] = cropValueDefault;// or return cropValueDefault;
}
return sprites[id];
};
} else if (sprites[iconImg] === undefined) {
const warning = `WARNING: "${iconImg}" not found in sprite file`;
if (!warn.has(warning)) {
warn.add(warning);
console.warn(warning);
}
style.icon.cropValues = cropValueDefault;
}
}
style.icon.source = sprites.source;
Expand Down Expand Up @@ -919,29 +961,28 @@ class Style {
* @param {Boolean} canBeFilled - true if feature.type == FEATURE_TYPES.POLYGON.
*/
applyToCanvasPolygon(txtrCtx, polygon, invCtxScale, canBeFilled) {
const context = this.context;
// draw line or edge of polygon
if (this.stroke) {
if (this.stroke.width > 0) {
// TO DO add possibility of using a pattern (https://github.com/iTowns/itowns/issues/2210)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

should we check for stroke properties too, like for fill ?

this._applyStrokeToPolygon(txtrCtx, invCtxScale, polygon, context);
this._applyStrokeToPolygon(txtrCtx, invCtxScale, polygon);
}

// fill inside of polygon
if (canBeFilled && this.fill) {
if (canBeFilled && (this.fill.pattern || this.fill.color)) {
// canBeFilled can be move to StyleContext in the later PR
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Could we remove canBeFilled ? Its value is feature.type == FEATURE_TYPES.POLYGON . We are in a method named applyToCanvasPolygon so aren't already expecting to handle only polygons ?

this._applyFillToPolygon(txtrCtx, invCtxScale, polygon, context);
this._applyFillToPolygon(txtrCtx, invCtxScale, polygon);
}
}

_applyStrokeToPolygon(txtrCtx, invCtxScale, polygon) {
if (txtrCtx.strokeStyle !== this.stroke.color) {
txtrCtx.strokeStyle = this.stroke.color;
}
const width = (this.stroke.width || 2.0) * invCtxScale;
const width = this.stroke.width * invCtxScale;
if (txtrCtx.lineWidth !== width) {
txtrCtx.lineWidth = width;
}
const alpha = this.stroke.opacity == undefined ? 1.0 : this.stroke.opacity;
const alpha = this.stroke.opacity;
if (alpha !== txtrCtx.globalAlpha && typeof alpha == 'number') {
txtrCtx.globalAlpha = alpha;
}
Expand All @@ -957,7 +998,7 @@ class Style {
// need doc for the txtrCtx.fillStyle.src that seems to always be undefined
if (this.fill.pattern) {
let img = this.fill.pattern;
const cropValues = this.fill.pattern.cropValues;
const cropValues = { ...this.fill.pattern.cropValues };
if (this.fill.pattern.source) {
img = await loadImage(this.fill.pattern.source);
}
Expand Down Expand Up @@ -1032,7 +1073,7 @@ class Style {
if (!this.icon.cropValues && !this.icon.color) {
icon.src = this.icon.source;
} else {
const cropValues = this.icon.cropValues;
const cropValues = { ...this.icon.cropValues };
const color = this.icon.color;
const id = this.icon.id || this.icon.source;
const img = await loadImage(this.icon.source);
Expand Down
48 changes: 32 additions & 16 deletions src/Layer/LabelLayer.js
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -248,9 +248,18 @@ class LabelLayer extends GeometryLayer {
context.setZoom(extentOrTile.zoom);

data.features.forEach((f) => {
// TODO: add support for LINE and POLYGON
if (f.style.text) {
if (Object.keys(f.style.text).length === 0) {
return;
}
}

if (f.type !== FEATURE_TYPES.POINT) {
return;
const warn = `Type label ${Object.keys(FEATURE_TYPES).filter(ft => FEATURE_TYPES[ft] === f.type)} not well supported`;
if (!this.source.warn.has(warn)) {
this.source.warn.add(warn);
console.warn(warn);
}
}
context.setFeature(f);

Expand All @@ -264,19 +273,11 @@ class LabelLayer extends GeometryLayer {
labels.needsAltitude = labels.needsAltitude || this.forceClampToTerrain === true || (isDefaultElevationStyle && !f.hasRawElevationData);

f.geometries.forEach((g) => {
// NOTE: this only works because only POINT is supported, it
// needs more work for LINE and POLYGON
coord.setFromArray(f.vertices, g.size * g.indices[0].offset);
// Transform coordinate to data.crs projection
coord.applyMatrix4(data.matrixWorld);

if (!_extent.isPointInside(coord)) { return; }
const geometryField = g.properties.style && g.properties.style.text && g.properties.style.text.field;

context.setGeometry(g);
let content;
this.style.setContext(context);
const layerField = this.style.text && this.style.text.field;
const geometryField = g.properties.style && g.properties.style.text && g.properties.style.text.field;
let content;
if (this.labelDomelement) {
content = readExpression(this.labelDomelement, context);
} else if (!geometryField && !featureField && !layerField) {
Expand All @@ -288,12 +289,27 @@ class LabelLayer extends GeometryLayer {
}
}

const label = new Label(content, coord.clone(), this.style);
if (this.style.zoom.min > this.style.context.zoom || this.style.zoom.max <= this.style.context.zoom) {
return;
}

// NOTE: this only works fine for POINT.
// It needs more work for LINE and POLYGON as we currently only use the first point of the entity

label.layerId = this.id;
label.padding = this.margin || label.padding;
g.indices.forEach((i) => {
coord.setFromArray(f.vertices, g.size * i.offset);
// Transform coordinate to data.crs projection
coord.applyMatrix4(data.matrixWorld);

labels.push(label);
if (!_extent.isPointInside(coord)) { return; }

const label = new Label(content, coord.clone(), this.style);

label.layerId = this.id;
label.padding = this.margin || label.padding;

labels.push(label);
});
});
});

Expand Down
32 changes: 17 additions & 15 deletions src/Parser/VectorTileParser.js
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -116,10 +116,11 @@ function vtFeatureToFeatureGeometry(vtFeature, feature, classify = false) {
function readPBF(file, options) {
options.out = options.out || {};
const vectorTile = new VectorTile(new Protobuf(file));
const sourceLayers = Object.keys(vectorTile.layers);
const vtLayerNames = Object.keys(vectorTile.layers);

if (sourceLayers.length < 1) {
return;
const collection = new FeatureCollection(options.out);
if (vtLayerNames.length < 1) {
return Promise.resolve(collection);
}

// x,y,z tile coordinates
Expand All @@ -130,26 +131,27 @@ function readPBF(file, options) {
// Only if the layer.origin is top
const y = options.in.isInverted ? options.extent.row : (1 << z) - options.extent.row - 1;

const collection = new FeatureCollection(options.out);

const vFeature = vectorTile.layers[sourceLayers[0]];
// TODO: verify if size is correct because is computed with only one feature (vFeature).
const size = vFeature.extent * 2 ** z;
const vFeature0 = vectorTile.layers[vtLayerNames[0]];
// TODO: verify if size is correct because is computed with only one feature (vFeature0).
const size = vFeature0.extent * 2 ** z;
const center = -0.5 * size;

collection.scale.set(globalExtent.x / size, -globalExtent.y / size, 1);
collection.position.set(vFeature.extent * x + center, vFeature.extent * y + center, 0).multiply(collection.scale);
collection.position.set(vFeature0.extent * x + center, vFeature0.extent * y + center, 0).multiply(collection.scale);
collection.updateMatrixWorld();

sourceLayers.forEach((layer_id) => {
if (!options.in.layers[layer_id]) { return; }
vtLayerNames.forEach((vtLayerName) => {
if (!options.in.layers[vtLayerName]) { return Promise.resolve(collection); }

const sourceLayer = vectorTile.layers[layer_id];
const vectorTileLayer = vectorTile.layers[vtLayerName];

for (let i = sourceLayer.length - 1; i >= 0; i--) {
const vtFeature = sourceLayer.feature(i);
for (let i = vectorTileLayer.length - 1; i >= 0; i--) {
const vtFeature = vectorTileLayer.feature(i);
vtFeature.tileNumbers = { x, y: options.extent.row, z };
const layers = options.in.layers[layer_id].filter(l => l.filterExpression.filter({ zoom: z }, vtFeature) && z >= l.zoom.min && z < l.zoom.max);
// Find layers where this vtFeature is used
const layers = options.in.layers[vtLayerName]
.filter(l => l.filterExpression.filter({ zoom: z }, vtFeature));

let feature;

for (const layer of layers) {
Expand Down
1 change: 1 addition & 0 deletions src/Source/Source.js
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -108,6 +108,7 @@ class Source extends InformationsData {
constructor(source) {
super(source);
this.isSource = true;
this.warn = new Set();

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Aggregating logs is a good idea and needed in itowns. I think we should not store aggregated logs in objects and pass them around as function parameters. An alternative implementation would be to implement a LogAggregator that stores logs, aggregates them and can log info, warn and errors. A global instance could be used by all itowns components then. If we do so, we need to be careful with memory usage and empty the log cache frequently. And if we don't want to implement it ourselves, I'm sure there are libraries available. However I think that's out of scope of this PR, so I'd be for removing this log aggregation for now and implement it differently later. @Desplandis what is your opinion on this?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah, we could have some kind of heap with a max size to aggregate those warnings. I think there should be some kind of library to do it. However, I agree that this is out of scope of this PR.

if (!source.url) {
throw new Error('New Source: url is required');
Expand Down
Loading