Do we need a new dot function?

[fys2045]

I recently discovered an issue when using the contourf function with hatches to fill significant regions and exporting vector format images from Python to insert into Word. Initially, everything appears fine, but when I export this Word document to a PDF, the areas filled with hatches turn into pixelated blocks, while other parts remain clear. This problem also occurs with GeoCAT when using hatches for filling. (Of course, this issue does not arise when not using vector images, but many journal submissions require vector graphics.) I am quite certain that there is a flaw with the hatches in the contourf function. Fortunately, NCL does not have this problem.

I am wondering if the GeoCAT team could draw inspiration from NCL's filling logic to develop a function in Python that can achieve significant dot filling. If this can be implemented, it would be highly beneficial for conducting meteorological research and publishing papers using GeoCAT.

I look forward to your response.

Best regards,

[fys2045]

[kafitzgerald]

Thanks for the note about this and sorry for the slow response!

I'll see if I can replicate this today and better understand where this problem might be coming from.

[jukent]

@fys2045 Could you provide us with the sample code to create these graphics?

[fys2045]

I am very willing to provide an example that can reproduce the previous problem, but it is not the program of the picture I provided before.
test.pdf
test.docx

import xarray as xr
import matplotlib.pyplot as plt

import geocat.datafiles as gdf
import geocat.viz as gv
# Read in data


# Open a netCDF data file using xarray default engine and load the data into xarrays
ds = xr.open_dataset(gdf.get('netcdf_files/atmos.nc'), decode_times=False)

# Select meridional wind at lowest pressure level
v = ds.V.isel(time=0, lev=0)
# Plot


# Generate figure (set its size (width, height) in inches)
plt.figure(figsize=(8, 10))
ax = plt.axes()

# Choose hatches to fill the contours
hatches = ['/////', '/////', '/////', '/////', None, '..', '.', '.', '.']

# Choose colors for the hatches
colors = [
    'coral', 'palegreen', 'royalblue', 'lemonchiffon', 'white', 'fuchsia',
    'brown', 'cyan', 'mediumblue'
]

# Create a filled contour plot
p = v.plot.contourf(ax=ax,
                    vmin=-45.0,
                    vmax=45,
                    levels=10,
                    add_colorbar=False,
                    hatches=hatches,
                    cmap='white')  # Use white cmap to have a white background

# Set the colors for the hatches
for i, collection in enumerate(p.collections):
    collection.set_edgecolor(colors[i % len(colors)])
    collection.set_linewidth(0.)

# Set linewidth of hatches
plt.rcParams['hatch.linewidth'] = 2.5

# Plot the contour lines
c = v.plot.contour(ax=ax,
                   vmin=-45.0,
                   vmax=45,
                   levels=10,
                   colors='k',
                   linewidths=1,
                   add_colorbar=False,
                   linestyles='solid')

# Add horizontal colorbar
cbar = plt.colorbar(p,
                    orientation='horizontal',
                    shrink=0.97,
                    aspect=10,
                    pad=0.09,
                    drawedges=True)
cbar.ax.tick_params(labelsize=16)
cbar.set_ticks(np.arange(-35, 40, 10))

# Color the hatches in colorbar
# We need to do this manually because matplotlib only uses information
# from the contourf call to create the colorbar.
for i, patch in enumerate(cbar.solids_patches):
    patch.set(edgecolor=colors[i % len(colors)])

# Use geocat-viz utility function to format latitude and longitude labels
gv.add_lat_lon_ticklabels(ax)

# Use geocat-viz utility function to format major and minor ticks
gv.add_major_minor_ticks(ax,
                         x_minor_per_major=2,
                         y_minor_per_major=3,
                         labelsize=16)

# Use geocat-viz utility function to set titles and labels
gv.set_titles_and_labels(ax,
                         maintitle="Patterned Contour Plot",
                         maintitlefontsize=18,
                         lefttitle="meridional wind component",
                         lefttitlefontsize=16,
                         righttitle="m/s",
                         righttitlefontsize=16)

# Remove default x and y labels
ax.set_xlabel(None)
ax.set_ylabel(None)

# Use geocat-viz utility function to set tick marks and tick labels
gv.set_axes_limits_and_ticks(
    ax,
    xticks=np.arange(0, 360, 60),
    yticks=np.arange(-60, 90, 30),
    xticklabels=['0', '60E', '120E', '180', '120W', '60W'],
    yticklabels=['60S', '30S', '0', '30N', '60N'])


plt.savefig("{}.svg".format('hello'),bbox_inches='tight')

plt.show()

[kafitzgerald]

I think I missed part of your description earlier.

It sounds like the issue you're seeing is happening following the export to PDF from an MS Word document.

I'm guessing something is causing the figure to poorly rasterize in this conversion process from a Word document to PDF. This rasterization shouldn't need to happen, but might be occurring depending on what version of Word you have, how it's configured, and how you're exporting to PDF. These troubleshooting tips might help.

I did try to replicate this issue (via saving a matplotlib figure with hatching using the SVG vector format, adding it to a Word document, and then exporting to PDF), but wasn't able to replicate the problem - it all seemed to work fine for me. I could very well have different versions though and/or things configured differently.

I hope this helps. Please let us know if not and share a short example with a bit more info about your system (as mentioned by Julia).

[fys2045]

I'm very sorry, but I need to add that using the print option to export to PDF will not cause the dots to turn into mosaics, but the images in the exported PDF will no longer be in vector format, causing them to become blurry. In other words, when I try to keep the images in the PDF in vector format, the dots change, which I find unacceptable.

To further explain my understanding of this issue, when I use the scatter function to create the dots and insert the image into Word before exporting it to PDF (either by saving as PDF or choosing export to PDF), the dots remain dots and do not turn into mosaics. So it seems to be an issue with the contourf function? At least that's how it appears at the moment.

I am willing to provide the code for the scatter function dots:

stepLAT = 1
stepLON = 1
area = np.where(p[::stepLAT, ::stepLON] < 0.1)
nx, ny = np.meshgrid(lon[::stepLON], lat[::stepLAT])
ax1.scatter(nx[area], ny[area], color='k', s=0.1, zorder=0, transform=ccrs.PlateCarree())

A word document containing an .svg image: SVG.docx
Use the Export PDF option to get a PDF file, and the dots become mosaics: one test.pdf
Use the print option to get a PDF file. The dots remain unchanged, but the image clarity is greatly reduced. This is because the image is no longer a vector image, but a bitmap image. : second test.pdf
A word document containing an svg image: SVG.docx

[jukent]

Hi @fys2045 I do not have access to Word on my work machine so I am unable to attempt to recreate the point where your images no longer appear as intended. However, it seems to me like if you look into matplotlib backends, and this SVG interpolation issue you might find a solution. It's possible that part of the plot is in vector graphics and part rasterized, check for consistency.

Alternatively, you could manipulate your figure size and dpi so that saving it as a non-SVG still is not blurry. When I saved it with plt.savefig('output.png', dpi=300) (because Google Docs does not support including SVGs) and then exported to a PDF, the graphics still look fine to me.
Test Dot Issue.pdf

[kafitzgerald]

@fys2045, I took a quick look at the files you shared and it does seem like part of your figure (the hatched portions) was poorly preserved in the conversion from Word to PDF on your machine. However, I still can't replicate this on my machine. When I save to PDF from the document you shared, the figure looks largely the same as the original. For reference, I have Microsoft Word for Mac - Version 16.86 installed. While I don't know what's causing the issue on your system, it seems like it's likely some sort of setting and/or lack of support for certain SVG features present in this file vs. the one(s) from NCL. I'd suggest reporting the issue to Microsoft.

In the meantime, here are a few more ideas to workaround this issue:

• Submitting figures separately in vector formats to the journal and inserting *.png versions into your Word document to avoid the conversion issue
• Saving to PDF from a different version of Microsoft Word
• You could also try upgrading or downgrading your version of matplotlib to see if there might be better compatibility for different versions

If you do find something is wrong or invalid in the SVG file itself generated with matplotlib/geocat-viz (using matplotlib), that issue should be reported to matplotlib.

While we really appreciate your suggestion here and the issue report, I think this out of scope for geocat-viz.

[fys2045]

@fys2045, I took a quick look at the files you shared and it does seem like part of your figure (the hatched portions) was poorly preserved in the conversion from Word to PDF on your machine. However, I still can't replicate this on my machine. When I save to PDF from the document you shared, the figure looks largely the same as the original. For reference, I have Microsoft Word for Mac - Version 16.86 installed. While I don't know what's causing the issue on your system, it seems like it's likely some sort of setting and/or lack of support for certain SVG features present in this file vs. the one(s) from NCL. I'd suggest reporting the issue to Microsoft.

In the meantime, here are a few more ideas to workaround this issue:

• Submitting figures separately in vector formats to the journal and inserting *.png versions into your Word document to avoid the conversion issue
• Saving to PDF from a different version of Microsoft Word
• You could also try upgrading or downgrading your version of matplotlib to see if there might be better compatibility for different versions

If you do find something is wrong or invalid in the SVG file itself generated with matplotlib/geocat-viz (using matplotlib), that issue should be reported to matplotlib.

While we really appreciate your suggestion here and the issue report, I think this out of scope for geocat-viz.

I am very sorry for taking up your time. Could you please provide the document converted from Word to PDF and the method you used to convert it to PDF? I would like to compare them carefully to find out where the problem may be. Thank you very much

[fys2045]

Hi @fys2045 I do not have access to Word on my work machine so I am unable to attempt to recreate the point where your images no longer appear as intended. However, it seems to me like if you look into matplotlib backends, and this SVG interpolation issue you might find a solution. It's possible that part of the plot is in vector graphics and part rasterized, check for consistency.

Alternatively, you could manipulate your figure size and dpi so that saving it as a non-SVG still is not blurry. When I saved it with plt.savefig('output.png', dpi=300) (because Google Docs does not support including SVGs) and then exported to a PDF, the graphics still look fine to me. Test Dot Issue.pdf

I agree with your point of view, maybe it is because these dotted places are not in vector format but in raster format. I tried to open it in AI (adobe illustrator2023) and found that these points have no contours, which seems to mean that these points are not in vector format, which may be related to the filling logic of contourf hatches. I will report this issue to the official matplotlib again.
Of course, this error has nothing to do with geocat-viz, but if the official matplotlib cannot solve this problem, will geocat-viz consider developing a better function to replace contourf hatches?