Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[EMAIL-207] Modify createDataSource method to use AttachmentDataSource #159

Closed

Conversation

violetbeach
Copy link
Contributor

Issue

Hello,

OOM occurred in the mail service developed using Apache-commons-email.

I debugged and found that the mail lookup function was wasting memory!

Problem

Below is a test for reproducing the problem.

@Test
void memoryTest() throws Exception {
    InputStream is = new SharedFileInputStream("/test/oom.eml");
    Session session = Session.getInstance(System.getProperties());
    MimeMessage mimeMessage = new MimeMessage(session, is);
    MimeMessageParser parser = new MimeMessageParser(mimeMessage).parse();
}

Below is the memory usage.

img

I expected to write attachments to the buffer of the output stream when absolutely necessary.

However, it is becoming a problem by putting all bytes[] in memory.

img_3

img_1

Sugest

So i modified these processes to use AttachmentDataSource.

Attachment DataSource uses InputStream from original DataSource to ensure memory is not wasted.

As a result, i can get a big effect in memory usage.

img_2

I worked hard to fix it. Feel free to let me know if you have any advice or lack!

If you need the tested eml file, I will send it to you by email.

Thanks very much. 🙇

@violetbeach violetbeach changed the title Modify createDataSource method to use AttachmentDataSource Email-207: Modify createDataSource method to use AttachmentDataSource Aug 7, 2023
Copy link
Member

@garydgregory garydgregory left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@violetbeach
Thank you for your PR.

  • All new public and protected elements should have a Javadoc comment
  • All new and changed code should have a matching unit test

@violetbeach
Copy link
Contributor Author

  • All new public and protected elements should have a Javadoc comment
  • All new and changed code should have a matching unit test

Thank very much!

I added Javadoc comments on public and protected elements. And I added a unit test for new class.

But, i could not cover the changed createDataSource method of MimeMessageParser because it is a difficult area to test.
I could only check that all the existing tests worked well.

Please let me know if there's anything missing! Thank you.

File ends in a blank line.
@codecov-commenter
Copy link

codecov-commenter commented Sep 29, 2023

Codecov Report

Merging #159 (cf8417c) into master (b5511b9) will increase coverage by 0.16%.
Report is 30 commits behind head on master.
The diff coverage is 100.00%.

@@             Coverage Diff              @@
##             master     #159      +/-   ##
============================================
+ Coverage     65.87%   66.03%   +0.16%     
- Complexity      305      309       +4     
============================================
  Files            18       19       +1     
  Lines          1052     1057       +5     
  Branches        138      137       -1     
============================================
+ Hits            693      698       +5     
  Misses          280      280              
  Partials         79       79              
Files Coverage Δ
...apache/commons/mail/util/AttachmentDataSource.java 100.00% <100.00%> (ø)
...rg/apache/commons/mail/util/MimeMessageParser.java 83.95% <100.00%> (-1.92%) ⬇️

📣 We’re building smart automated test selection to slash your CI/CD build times. Learn more

@@ -0,0 +1,104 @@
/*
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Oh! It's almost the same.

asfgit pushed a commit that referenced this pull request Dec 10, 2023
Tests InputStreamDataSource based on patch #159 from Lee Jaeheon and
modified
@garydgregory garydgregory changed the title Email-207: Modify createDataSource method to use AttachmentDataSource [EMAIL-207] Modify createDataSource method to use AttachmentDataSource Dec 10, 2023
@garydgregory
Copy link
Member

Hello @violetbeach
This issue has been handled slightly differently in git master. You are credited in changes.xml.

@violetbeach
Copy link
Contributor Author

Hello @garydgregory

I've verified that the issue has been solved!

image

Memory is much cleaner, and the parser no longer puts the entire data in the file into memory.

Thank you. 👍

@garydgregory
Copy link
Member

@violetbeach 👍

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants