ArgumentError: invalid byte sequence in UTF-8

[stef157]

Describe the bug

When sending a large attachment (30M), I get this message.
If I send a small one, there is no problem.

To Reproduce

1. Send an email with an attachment

Expected behaviour

The mail si send

Postal Environment details

• OS: Debian 12
• Version: 3.2.2

Environment details

• Type: Printer

Additional information/context

`2024-10-15 13:36:02 +0000 ERROR An error occurred while processing data from a client. trace_id=VMKGZU2Y component=smtp-server

2024-10-15 13:36:02 +0000 ERROR ArgumentError: invalid byte sequence in UTF-8 trace_id=VMKGZU2Y component=smtp-server

2024-10-15 13:36:02 +0000 ERROR /opt/postal/app/lib/postal/message_db/database.rb:329:in 'query_on_connection' trace_id=VMKGZU2Y component=smtp-server

2024-10-15 13:36:02 +0000 ERROR /opt/postal/app/lib/postal/message_db/database.rb:234:in 'block in insert' trace_id=VMKGZU2Y component=smtp-server

2024-10-15 13:36:02 +0000 ERROR /opt/postal/app/lib/postal/message_db/connection_pool.rb:20:in 'use' trace_id=VMKGZU2Y component=smtp-server

2024-10-15 13:36:02 +0000 ERROR /opt/postal/app/lib/postal/message_db/database.rb:345:in 'with_mysql' trace_id=VMKGZU2Y component=smtp-server

2024-10-15 13:36:02 +0000 ERROR /opt/postal/app/lib/postal/message_db/database.rb:233:in 'insert' trace_id=VMKGZU2Y component=smtp-server

2024-10-15 13:36:02 +0000 ERROR /opt/postal/app/lib/postal/message_db/database.rb:126:in 'insert_raw_message' trace_id=VMKGZU2Y component=smtp-server

2024-10-15 13:36:02 +0000 ERROR /opt/postal/app/lib/postal/message_db/message.rb:278:in 'save_raw_message' trace_id=VMKGZU2Y component=smtp-server

2024-10-15 13:36:02 +0000 ERROR /opt/postal/app/lib/postal/message_db/message.rb:208:in 'save' trace_id=VMKGZU2Y component=smtp-server

2024-10-15 13:36:02 +0000 ERROR /opt/postal/app/app/lib/smtp_server/client.rb:506:in 'block in finished' trace_id=VMKGZU2Y component=smtp-server

2024-10-15 13:36:02 +0000 ERROR /opt/postal/app/app/lib/smtp_server/client.rb:490:in 'each' trace_id=VMKGZU2Y component=smtp-server

2024-10-15 13:36:02 +0000 ERROR /opt/postal/app/app/lib/smtp_server/client.rb:490:in 'finished' trace_id=VMKGZU2Y component=smtp-server

2024-10-15 13:36:02 +0000 ERROR /opt/postal/app/app/lib/smtp_server/client.rb:429:in 'block in data' trace_id=VMKGZU2Y component=smtp-server

2024-10-15 13:36:02 +0000 ERROR /opt/postal/app/app/lib/smtp_server/client.rb:72:in 'handle' trace_id=VMKGZU2Y component=smtp-server

2024-10-15 13:36:02 +0000 ERROR /opt/postal/app/app/lib/smtp_server/server.rb:204:in 'block (2 levels) in run_event_loop' trace_id=VMKGZU2Y component=smtp-server

2024-10-15 13:36:02 +0000 ERROR /opt/postal/app/app/lib/smtp_server/server.rb:106:in 'select' trace_id=VMKGZU2Y component=smtp-server

2024-10-15 13:36:02 +0000 ERROR /opt/postal/app/app/lib/smtp_server/server.rb:106:in 'block in run_event_loop' trace_id=VMKGZU2Y component=smtp-server

2024-10-15 13:36:02 +0000 ERROR /opt/postal/app/app/lib/smtp_server/server.rb:104:in 'loop' trace_id=VMKGZU2Y component=smtp-server

2024-10-15 13:36:02 +0000 ERROR /opt/postal/app/app/lib/smtp_server/server.rb:104:in 'run_event_loop' trace_id=VMKGZU2Y component=smtp-server

2024-10-15 13:36:02 +0000 ERROR /opt/postal/app/app/lib/smtp_server/server.rb:39:in 'block in run' trace_id=VMKGZU2Y component=smtp-server

2024-10-15 13:36:02 +0000 ERROR /usr/local/bundle/gems/klogger-logger-1.4.0/lib/klogger/group_set.rb:18:in 'call_without_id' trace_id=VMKGZU2Y component=smtp-server

2024-10-15 13:36:02 +0000 ERROR /usr/local/bundle/gems/klogger-logger-1.4.0/lib/klogger/logger.rb:72:in 'tagged' trace_id=VMKGZU2Y component=smtp-server

2024-10-15 13:36:02 +0000 ERROR /opt/postal/app/app/lib/smtp_server/server.rb:37:in 'run' trace_id=VMKGZU2Y component=smtp-server

2024-10-15 13:36:02 +0000 ERROR script/smtp_server.rb:13:in '

' trace_id=VMKGZU2Y component=smtp-server`

[willpower232]

Did you increase the maximum size of the messages in the config file?

https://github.com/postalserver/postal/pull/383/files

Also unrelated but probably worth an update of your Postal to 3.3.4

[stef157]

I’ve tried with with a simple mail and it’s working (30-40M)
Also the printer can send the mail if there is not too much pages

[stef157]

Also, when the problem As the size I was getting a specific message.

[willpower232]

So basically it is only the printer that has a problem, you can send messages of that size? There might be a problem with the printer or it simply cannot do it.

[stef157]

Hi @willpower232 ,

I switched to using Postfix without any layer on top, and everything is going well. From there, I analyzed the logs more closely, and the problem seems to be — I believe — that the content is being sent in 7-bit encoding. Unfortunately, it is not possible to adjust the encoding at the printer level.

This would explain why the size is not actually the cause, but rather a potential string of characters that gets generated depending on the scanned document… and this string of characters is more likely to occur in a large document than a smaller one.

[willpower232]

Ah that makes sense, glad you figured it out 🙏