Support for TFT and touch screen - need feedback and testing

[jmz52]

TFT screens with optional touch panel can be used to control Marlin
Low-level IO uses DMA for background data transfer to TFT screen to minimize impact on Marlin's performance.

Current low-level IO implementation is for HAL STM32 and MCU STM32F1 only.
SPI implementation lacks STM32F4 initialization code.
FSMC implementation STM32F4 initialization code and (temporary) hardcoded to Zx specific.
Implementation for STM32F4 and FSMC on VE chips will follow.

Implemented Marlin_UI class for 320x240 screen.
Can be used with or without rotary encoder.
Support UBL or advanced pause is not implemented yet.

Tested on MKS Robin v2.4 with TFT v2.0 (ST7789v controller)

Earlier builds were tested on EZT-T1 and SKR Mini v1.1 with 128x128 (st7735 controller) but UI 128x128 became outdated due to changes made during touch support development.
UI 128x128 implementation will be uploaded once code is updated.

SPI IO required dedicated hardware SPI interface wich can't be shared with SD card. Should be OK to use same SPI for TFT screen and touch panel.

Build environment for MKS Robin - MarlinFirmware/Configurations#114
https://github.com/jmz52/Configurations/tree/mks_robin_hal_stm32/config/examples/Mks/Robin_HAL_STM32

[jmz52]

Bootscreen

[jmz52]

Status screen
Do not mind grey color of hotends - there are no sensors connected.
Hotends, heated bed and chamber will change colors when they are hot.
Heated bed and heated chamber will change icons when heater is active.

Hotends, heated bed , heated chamber, fan, feed rate, flow rate are clickable and will lead to corresponding edit screens
Settings icon will lead to main menu.
SD card will lead to "print from sd" menu when sd is mounted and printer is idle.

[jmz52]

Menu screens
With page up, back and page down clickable navigation icons.
Click to menu item to select it.
Click to selected icon to activate it.
Double click logic is used to reduce errors when dealing with smaller screens.

[jmz52]

Edit screens
With decrease, confirm, increase and slider controls.
Axis move screen does not have slider due to specific encoder handling.

[jmz52]

Confirm screen
With cancel and confirm controls.
Selected control is highlighted and another one is dimmed, so you can know with one is selected if you use encoder instead of touch panel.

[jmz52]

Kill screen

[jmz52]

Related issues
#16787
#17844
#16732

[jmz52]

@hobiseven, would you mind some code adoption and testing on Alfawise Ux0 boards?

[hobiseven]

Gosh... Looks like we do the same thing, different flavors. I have a Little LGVL glue ported and running on Alfawise...
See what we are about to test :

https://www.lesimprimantes3d.fr/forum/uploads/monthly_2020_05/IMG_1702.jpg.2878a786f88dc99978ec135913587ea4.jpg

I invite you to read the code I built upon tag 2.5.0.3.
This is work in progress. I use extui API.

I send you an invite to my privare repo..

[hobiseven]

@jmz52
https://www.lesimprimantes3d.fr/forum/uploads/monthly_2020_05/IMG_1691.jpg.c92379445e9526c18adb49b1478a7ba2.jpg

LVGL is very nice to use, and the adaptation to get DMAs running in the background is really tiny.

[hobiseven]

@jmz52 something which LVGL allows : dynamic displays : See the small moovie. I get the code from one of the forum member tonight . Please check the small video below :

https://www.lesimprimantes3d.fr/forum/applications/core/interface/file/attachment.php?id=92278

[thisiskeithb]

Did you see MKS’ LVGL port? #18071

[hobiseven]

NOOOPEEE. So we all on our side do work on the same things!

[thisiskeithb]

Options are nice 🙂

[mattdog01]

This is beautiful. Absolutely love it.
However, are you going to make it work for TFT v1.1?
In the current nightly build the "touch button" don't work on my TFT v1.1. They are there but no action. Will this upgrade fix that?

[jmz52]

@mattdog01 TFT v1.1 needs different calibration data.
Check you Configuration.h for code below to see if you are using right numbers.
Default are for TVT v2.0

New code should work on TFT v1.1, but you need to change calibration data in
https://github.com/jmz52/Marlin/blob/tft_and_touch_screen/Marlin/src/HAL/STM32/tft/xpt2046.h

And make sure you are using build environment from
https://github.com/jmz52/Configurations/tree/mks_robin_hal_stm32/config/examples/Mks/Robin_HAL_STM32
Check extra_config.ini for details

//
// ADS7843/XPT2046 ADC Touchscreen such as ILI9341 2.8
//
#define TOUCH_BUTTONS
#if ENABLED(TOUCH_BUTTONS)
  #define BUTTON_DELAY_EDIT  75 // (ms) Button repeat delay for edit screens
  #define BUTTON_DELAY_MENU 100 // (ms) Button repeat delay for menus

  /* MKS Robin TFT v2.0 */
  #define XPT2046_X_CALIBRATION    12013
  #define XPT2046_Y_CALIBRATION    -8711
  #define XPT2046_X_OFFSET           -32
  #define XPT2046_Y_OFFSET           256

  /* MKS Robin TFT v1.1 */
  //#define XPT2046_X_CALIBRATION -11792
  //#define XPT2046_Y_CALIBRATION   8947
  //#define XPT2046_X_OFFSET         342
  //#define XPT2046_Y_OFFSET         -19
#endif

[hobiseven]

@mattdog01 looks like your touch settings are inverted in y. So touch zones are on the opposite side of the screen. We actually have also 3 touch screen flavors on alfawise.

[hobiseven]

@jmz52 I like your print screen. I was more thinking about more arrows for move and less text menus....

[MS1987]

@jmz52 @hobiseven , can I have your email or please send me email ? I want to develop the touch screen together.
My email: [email protected]

[hobiseven]

Folks,
I probably do not have the expertise and level you guys have... I share my repo with you also @MS1987

I will include a piece of LVGL code ( for the images / screens in the coming days, and push it. The goal for me was to make a POC of what the CPU can handle, without destroying the print performance. I started to build a small state machine , and picket icons googling here and there. @sensei73 asked to copy Alfawise U30 Pro GUI and he started to put together LVGL code to do that.

As lvgl only updates what is nedded, we already do less DMA than with DOGM if I am correct.

[looxonline]

Keen to see this ported to the LP1768/9 HALS. Will open it up substantially. Very nice to see marlin supporting TFT'S natively instead of via gcode as a proxy.

[hobiseven]

Let me ask one of my application manager if he can assign someone to this. I am from NXP.

[looxonline]

Let me ask one of my application manager if he can assign someone to this. I am from NXP.

That would be incredible. If the SPI mode is used it could open the functionality up to many thousands who use the SKR series of motherboards via the EXP2 header.

[hobiseven]

Well, you have a couple of things here :
LVGL is hardware independent, and beside the speed , all LVGL code could work on ANY TFT. The right LVGL hal has to be done. and this is far from rocket science. This includes FSMC for ST, SPI for a bunch or other boards, with speed limitations , as well as parallel TFT interfaces, for instance, on LPC1788.
Hence, it looks to me that LVGL is the proper vehicule going forward. @thinkyhead do you have any preference on the graphics lib?

[hobiseven]

Bootscreen

Hi @jmz52

How did you store that picture in the chip? Is that a jpg decompiled on the fly? or a SD card file?
3202402bytes/RGB565 is 153Kb, Jpeg is about 25Kb... hence my question

[mattdog01]

Hello @jmz52
I have a functional TFT v1.1 again. Thank you!
My Nightly Build downloads didn't have src/HAL/STM32/tft so I added the TFT folder to STM32. Then I was able to update /xpt2046.h with my screens calibration numbers. You had taught me about the calibration difference for v1.1 for the Configuration.h file and it has worked great for a few of the Marlin updates and then it stopped working. I had to use a previously compiled ROBIN.bin to restore printer function.
Prior to your help yesterday I had to revert back to "Marlin-MKS-Robin-ILI9328", that you provided me a while ago, just to be able to change settings on my printer and have it work.
So now I am really looking forward to setting up for your awesome GUI.
Did your reply give me enough info to get the new TFT function working on my printer? I tried to follow through it (but it was late in the day) and I had compiling errors that prevented a successful build. I will try again today.
Thanks so much for your work on this. It is going to be great to have the TFT experience again but under Marlin. I'm loving Marlin for machine functionality but I have missed the bling of the MKS firmware. Take care.

[jmz52]

I think some explanation is on order.
Originally I've started TFT work for EZT-T1 delta printer.
It has mainboard is based on STM32F103RBT6 MCU and 128x128 TFT screen based on st7735 controller.
MCU has 128KB flash only meaning Marlin with delta math and HAL_STM32F1 will never fit into it.
In order to squeeze firmware into such limited flash I've had to switch to HAL_STM32 and abandon u8glib completely.
Fonts and icons were reused, but one does not need an whole library to draw 1bpp images.
Instead of expanding existing 12864 graphical interface I've made a new implementation of MarlinUI class dedicated for color screen of particular resolution (128x128).
This approach is less flexible that extui, but is has an undeniable benefit of already developed menu system.
I am no expert in Marlin's internals, so it would be a tremendous amount of work for me to develop new menu structure.
Beside I am not sure how much flexibility is needed for tiny 128x128 screen.

With MarlinUI class implementation, once you have a working status screen you need to implement about 5 drawing functions to get a fully functional menu.
It is not be as pretty as UI used on MKS boards or BTT smart screen controllers, but it is way better that x2 upscaled 12864 interface we have on Robin boards for some time now.

Final result was rather satisfying - I was able to fit delta math, SD support, eeprom emulation and code for TFT screen into 128KB.

With new UI working it was a matter of several night to implements new IO subsystem for FSMC and get new UI to MKS Robin TFT. Then I could start working on new display resolution and touch support.

I am no familiar with LVGL yet, but if it is lightweight enough to not impact print quality it might be a good foundation for new UI. My only concern here is that it might be too flash consuming for low-end MCUs.

For now I am going to focus on SDIO issues and FSMC support for STM32F103VE and STM32F4 MCUs. This low-level functionality will be needed for either UI implementation.

[jmz52]

How did you store that picture in the chip? Is that a jpg decompiled on the fly? or a SD card file?
320_240_2bytes/RGB565 is 153Kb, Jpeg is about 25Kb... hence my question

@hobiseven this is raw 16-bit TFT image. There is an external SPI flash on MKS Robin board but I am not using it (yet)
https://github.com/jmz52/Marlin/blob/tft_and_touch_screen/Marlin/src/lcd/tft/images/bootscreen_320x240x16.h
Other images are in 4 bpp greyscale format colored with dynamically generated palette.
1 bpp and 2 bpp images are supported as well.

[sensei73]

@jmz52 Interesting, if we have access to the external SPI flash, we could use it to load images as menu. This technique is used on the U30 Pro, less ram consumed.

[jmz52]

@sjasonsmith, Black F407VE works fine.

[eghbalmohsen]

Hello @jmz52
Thanks for your work here, could you please guide me on how to use MKS Robin Nano v1.2 + MKS TFT32 display?
What should I change in the code in order to get things work on my 3.2 inch display?

[MsnSazabi]

@jmz52

Can you share bin files? Platformio cant compile code for black F407ve. Not from marlin bugfix nor from your fork.

[jmz52]

@MsnSazabi, code I am using can be found at https://github.com/jmz52/Marlin/tree/tft_and_touch_N2
In compiles just fine in VSCode+PlatformIO.

[MsnSazabi]

Ok, got screen running. But no reaction on touch. Running M995 from host starts calibration script, but it just dont react on touches. Touch screen shares same spi (1) with flash, cs pin set correctly. Can't understand why it doesnt work. Touch itself is ok, worked with native mks firmware.

[jmz52]

@MsnSazabi, what board and screen you have and how are they connected to each other?

[MsnSazabi]

@jmz52 mks robin2 with mks robin tft35 v1.0 (it's a set sold together). Board has stm32f407zet6. Screen connected via fsmc, touch on spi1 (sharing with spi flash). LCD driver is ili9488, touch is hr2046.

[jmz52]

MKS Robin (F103ZE) uses SPI2, try this configuration for touch screen

#define TOUCH_CS_PIN                      PB1   // SPI2_NSS
#define TOUCH_SCK_PIN                     PB13  // SPI2_SCK
#define TOUCH_MISO_PIN                    PB14  // SPI2_MISO
#define TOUCH_MOSI_PIN                    PB15  // SPI2_MOSI
#define TOUCH_INT_PIN                     -1

[MsnSazabi]

Nope, doesnt work. Actually, there is a scheme with wiring. So I'm sure touch goes to spi1.

@jmz52 is there any special definitions for touch screen? Maybe I'm missing some thing in code

[bkiziuk]

I have just moved to the newest bugfix-2.0.x and enabled the new UI to check it out.
I'm using MKS Robin Nano 1.2 with its stock TFT32 display.
The UI works after touch calibration but I have a question about the way menu should behave.
Currently, a single touch on Info Screen's Settings buttom takes to the menu. To choose the option from the menu, the first touch on menu item just highlights it. The second touch on this menu item then executes the command - so basically I need to touch the same option twice to make it active. Is it designed to be like this?

[jmz52]

Yes, it is designed this way to reduce number of errors on smaller touch screens.

[thisiskeithb]

@jmz52 Now that ColorUI is merged into Marlin, can this be closed?

[ugers]

@thisiskeithb 128x128 UI not implented, so i cannot check the functionality of the TFT st7735 display.
#16787
@jmz52 Any progress about 128x128 UI?

[Felipe-NS]

Yes, it is designed this way to reduce number of errors on smaller touch screens.

Where can I change this 2 default clicks to just one?(I use a bigger screen)

[eghbalmohsen]

I have just moved to the newest bugfix-2.0.x and enabled the new UI to check it out.
I'm using MKS Robin Nano 1.2 with its stock TFT32 display.
The UI works after touch calibration but I have a question about the way menu should behave.
Currently, a single touch on Info Screen's Settings buttom takes to the menu. To choose the option from the menu, the first touch on menu item just highlights it. The second touch on this menu item then executes the command - so basically I need to touch the same option twice to make it active. Is it designed to be like this?

Hi @bkiziuk
I am using the same hardware as yours, could you please share your bin files with me? I can't compile the code in VS Code.
I really appreciate your help.

[bkiziuk]

@eghbalmohsen not a problem for me
Robin_nano35.zip
However I'm not sure if it will be really useful for you unless you have the same hardware configuration. I'm using BLtouch for Zmin. If you have different setup, it will probably not work as you expect it to.

[eghbalmohsen]

@eghbalmohsen not a problem for me
Robin_nano35.zip
However I'm not sure if it will be really useful for you unless you have the same hardware configuration. I'm using BLtouch for Zmin. If you have different setup, it will probably not work as you expect it to.

@bkiziuk
Thanks for sharing the file. It didn't work for me, display shows a blue screen and mixed character.
Btw, could you please guide me which source you are compiling from? and What is your main configs? (not printer configs)
I just want to get this Robin nano v1.2 and 3.2" TFT_v2.0 to work.
Again, thanks for your help and guidance in this manner.

[bkiziuk]

@eghbalmohsen https://github.com/bkiziuk/Marlin/tree/bugfix-2.0.x

If you have 3.2" display, maybe you should play with //#define MKS_ROBIN_TFT32 option in Configuration.h

[eghbalmohsen]

@eghbalmohsen https://github.com/bkiziuk/Marlin/tree/bugfix-2.0.x

If you have 3.2" display, maybe you should play with //#define MKS_ROBIN_TFT32 option in Configuration.h

@bkiziuk
Wow, It worked. God bless you, I am really grateful of you bro.
The final thing is that touch screen is not calibrated. How can I calibrate touch screen?

Again, Thank you so much for your support, way better than MKS.

[thisiskeithb]

The final thing is that touch screen is not calibrated. How can I calibrate touch screen?

Send M995 via serial.

---

For best results getting help with configuration and troubleshooting, please use the following resources:

• MarlinFW.org Marlin Documentation
• RepRap.org Marlin Forum
• Tom's 3D Forums
• Facebook Group "Marlin Firmware"
• Facebook Group "Marlin Firmware for 3D Printers"
• Marlin Configuration on YouTube
• Marlin Discord server. Join link: https://discord.gg/n5NJ59y

[MsnSazabi]

Can someone advise me on setting hardware spi control register via marlin? My board uses hardware spi1 for touch screen, and via pullup resistors. As I understand, I need to set spi mode 3. But according to readed data, marlin doesnt change control register.

[thisiskeithb]

Can someone advise me on setting hardware spi control register via marlin? My board uses hardware spi1 for touch screen, and via pullup resistors. As I understand, I need to set spi mode 3. But according to readed data, marlin doesnt change control register.

Please use the links in my previous comment for configuration issues.

[eghbalmohsen]

The final thing is that touch screen is not calibrated. How can I calibrate touch screen?

Send M995 via serial.

For best results getting help with configuration and troubleshooting, please use the following resources:

• MarlinFW.org Marlin Documentation
• RepRap.org Marlin Forum
• Tom's 3D Forums
• Facebook Group "Marlin Firmware"
• Facebook Group "Marlin Firmware for 3D Printers"
• Marlin Configuration on YouTube
• Marlin Discord server. Join link: https://discord.gg/n5NJ59y

Thanks for your reply,
Do you know which serial port should I config in #define SERIAL_PORT ?
I connect the USB cable to my computer and nothing happens. It doesn't even recognize CH340 module.
Any ideas?

[DanJunior78]

Hi, can someone advice me how to get a custom bootscreen shown? I´ve a few MKS TS35 running on my printers. I´ve checked some code and the Marlin folders to follow the code.
How does it decide the used file (found the folder with the different .png files) or is it hard coded as bootscreens before?
Did i miss something checking all the messages in Github for it (i would be sorry for a duplicate)?
My printers show the smallest PNG during boot phase. There is a nice 480x320 png which looks awesome.

[thisiskeithb]

For best results getting help with configuration and troubleshooting, please use the following resources:

• MarlinFW.org Marlin Documentation
• RepRap.org Marlin Forum
• Tom's 3D Forums
• Facebook Group "Marlin Firmware"
• Facebook Group "Marlin Firmware for 3D Printers"
• Marlin Configuration on YouTube
• Marlin Discord server. Join link: https://discord.gg/n5NJ59y

[eghbalmohsen]

@bkiziuk I am using your source code, I could compile it successfully, thanks for your work bro.
Now I have a problem, I can't connect to serial port, what is the correct config for #define SERIAL_PORT ???
Any ideas?

[thisiskeithb]

@eghbalmohsen PLEASE stop bumping this thread for support issues and use the links I posted above for further configuration issues.