Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Caused by: java.nio.channels.ClosedChannelException #7185

Open
sysmat opened this issue Nov 24, 2024 · 9 comments
Open

Caused by: java.nio.channels.ClosedChannelException #7185

sysmat opened this issue Nov 24, 2024 · 9 comments
Labels
bug unable to reproduce

Comments

@sysmat
Copy link

sysmat commented Nov 24, 2024

Describe the bug

  • maven task fails
  • 4000 lines of error stack

Version of dependency-check used

  • 10.0.4 - 11.1.0
  • maven plugin
  • maven 3.9.9
  • on desktop & on CI

Log file

Caused by: org.h2.jdbc.JdbcSQLNonTransientException: General error: "org.h2.mvstore.MVStoreException: Reading from file sun.nio.ch.FileChannelImpl@171feab7 failed at 27941644 (length -1), read 0, remaining 768 [2.1.214/1]"; SQL statement:
SELECT id FROM VULNERABILITY CVE WHERE cve=? [50000-214]
    at org.h2.message.DbException.getJdbcSQLException (DbException.java:554)
    at org.h2.message.DbException.getJdbcSQLException (DbException.java:477)
    at org.h2.message.DbException.get (DbException.java:212)
    at org.h2.message.DbException.convert (DbException.java:395)
    at org.h2.command.Command.executeQuery (Command.java:211)
    at org.h2.jdbc.JdbcPreparedStatement.executeQuery (JdbcPreparedStatement.java:128)
    at org.owasp.dependencycheck.data.nvdcve.H2Functions.updateVulnerability (H2Functions.java:265)
    at jdk.internal.reflect.GeneratedMethodAccessor165.invoke (Unknown Source)
    at jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke (DelegatingMethodAccessorImpl.java:43)
    at java.lang.reflect.Method.invoke (Method.java:566)
    at org.h2.schema.FunctionAlias$JavaMethod.execute (FunctionAlias.java:495)
    at org.h2.schema.FunctionAlias$JavaMethod.getTableValue (FunctionAlias.java:363)
    at org.h2.expression.function.table.JavaTableFunction.getValue (JavaTableFunction.java:34)
    at org.h2.table.FunctionTable.getResult (FunctionTable.java:51)
    at org.h2.index.VirtualConstructedTableIndex.find (VirtualConstructedTableIndex.java:38)
    at org.h2.index.IndexCursor.find (IndexCursor.java:161)
    at org.h2.table.TableFilter.next (TableFilter.java:394)
    at org.h2.command.query.Select$LazyResultQueryFlat.fetchNextRow (Select.java:1832)
    at org.h2.result.LazyResult.hasNext (LazyResult.java:78)
    at org.h2.result.FetchedResult.next (FetchedResult.java:34)
    at org.h2.command.query.Select.queryFlat (Select.java:728)
    at org.h2.command.query.Select.queryWithoutCache (Select.java:833)
    at org.h2.command.query.Query.queryWithoutCacheLazyCheck (Query.java:197)
    at org.h2.command.query.Query.query (Query.java:512)
    at org.h2.command.query.Query.query (Query.java:475)
    at org.h2.command.CommandContainer.query (CommandContainer.java:251)
    at org.h2.command.Command.executeQuery (Command.java:190)
    at org.h2.jdbc.JdbcPreparedStatement.executeQuery (JdbcPreparedStatement.java:128)
    at org.apache.commons.dbcp2.DelegatingPreparedStatement.executeQuery (DelegatingPreparedStatement.java:123)
    at org.apache.commons.dbcp2.DelegatingPreparedStatement.executeQuery (DelegatingPreparedStatement.java:123)
    at org.owasp.dependencycheck.data.nvdcve.CveDB.updateOrInsertVulnerability (CveDB.java:1396)
    at org.owasp.dependencycheck.data.nvdcve.CveDB.updateVulnerability (CveDB.java:1100)
    at org.owasp.dependencycheck.data.update.nvd.api.NvdApiProcessor.updateCveDb (NvdApiProcessor.java:119)
    at org.owasp.dependencycheck.data.update.nvd.api.NvdApiProcessor.call (NvdApiProcessor.java:96)
    at org.owasp.dependencycheck.data.update.nvd.api.NvdApiProcessor.call (NvdApiProcessor.java:40)
    at java.util.concurrent.FutureTask.run (FutureTask.java:264)
    at java.util.concurrent.ThreadPoolExecutor.runWorker (ThreadPoolExecutor.java:1128)
    at java.util.concurrent.ThreadPoolExecutor$Worker.run (ThreadPoolExecutor.java:628)
    at java.lang.Thread.run (Thread.java:829)
Caused by: org.h2.mvstore.MVStoreException: Reading from file sun.nio.ch.FileChannelImpl@171feab7 failed at 27941644 (length -1), read 0, remaining 768 [2.1.214/1]
    at org.h2.mvstore.DataUtils.newMVStoreException (DataUtils.java:1004)
    at org.h2.mvstore.DataUtils.readFully (DataUtils.java:470)
    at org.h2.mvstore.FileStore.readFully (FileStore.java:98)
    at org.h2.mvstore.Chunk.readBufferForPage (Chunk.java:422)
    at org.h2.mvstore.MVStore.readPage (MVStore.java:2569)
    at org.h2.mvstore.MVMap.readPage (MVMap.java:633)
    at org.h2.mvstore.Page$NonLeaf.getChildPage (Page.java:1125)
    at org.h2.mvstore.CursorPos.traverseDown (CursorPos.java:61)
    at org.h2.mvstore.Cursor.traverseDown (Cursor.java:169)
    at org.h2.mvstore.Cursor.<init> (Cursor.java:40)
    at org.h2.mvstore.MVMap.cursor (MVMap.java:731)
    at org.h2.mvstore.tx.TransactionMap$TMIterator.<init> (TransactionMap.java:1088)
    at org.h2.mvstore.tx.TransactionMap$CommittedIterator.<init> (TransactionMap.java:935)
    at org.h2.mvstore.tx.TransactionMap.chooseIterator (TransactionMap.java:852)
    at org.h2.mvstore.tx.TransactionMap.keyIterator (TransactionMap.java:814)
    at org.h2.mvstore.db.MVSecondaryIndex.find (MVSecondaryIndex.java:276)
    at org.h2.mvstore.db.MVSecondaryIndex.find (MVSecondaryIndex.java:270)
    at org.h2.index.IndexCursor.find (IndexCursor.java:161)
    at org.h2.table.TableFilter.next (TableFilter.java:394)
    at org.h2.command.query.Select$LazyResultQueryFlat.fetchNextRow (Select.java:1832)
    at org.h2.result.LazyResult.hasNext (LazyResult.java:78)
    at org.h2.result.FetchedResult.next (FetchedResult.java:34)
    at org.h2.command.query.Select.queryFlat (Select.java:[728](https://git.arnes.si/portal-hub/db-hub-api/aris/-/jobs/1017699#L728))
    at org.h2.command.query.Select.queryWithoutCache (Select.java:833)
    at org.h2.command.query.Query.queryWithoutCacheLazyCheck (Query.java:197)
    at org.h2.command.query.Query.query (Query.java:512)
    at org.h2.command.query.Query.query (Query.java:475)
    at org.h2.command.CommandContainer.query (CommandContainer.java:251)
    at org.h2.command.Command.executeQuery (Command.java:190)
    at org.h2.jdbc.JdbcPreparedStatement.executeQuery (JdbcPreparedStatement.java:128)
    at org.owasp.dependencycheck.data.nvdcve.H2Functions.updateVulnerability (H2Functions.java:265)
    at jdk.internal.reflect.GeneratedMethodAccessor165.invoke (Unknown Source)
    at jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke (DelegatingMethodAccessorImpl.java:43)
    at java.lang.reflect.Method.invoke (Method.java:566)
    at org.h2.schema.FunctionAlias$JavaMethod.execute (FunctionAlias.java:495)
    at org.h2.schema.FunctionAlias$JavaMethod.getTableValue (FunctionAlias.java:363)
    at org.h2.expression.function.table.JavaTableFunction.getValue (JavaTableFunction.java:34)
    at org.h2.table.FunctionTable.getResult (FunctionTable.java:51)
    at org.h2.index.VirtualConstructedTableIndex.find (VirtualConstructedTableIndex.java:38)
    at org.h2.index.IndexCursor.find (IndexCursor.java:161)
    at org.h2.table.TableFilter.next (TableFilter.java:394)
    at org.h2.command.query.Select$LazyResultQueryFlat.fetchNextRow (Select.java:1832)
    at org.h2.result.LazyResult.hasNext (LazyResult.java:78)
    at org.h2.result.FetchedResult.next (FetchedResult.java:34)
    at org.h2.command.query.Select.queryFlat (Select.java:728)
    at org.h2.command.query.Select.queryWithoutCache (Select.java:833)
    at org.h2.command.query.Query.queryWithoutCacheLazyCheck (Query.java:197)
    at org.h2.command.query.Query.query (Query.java:512)
    at org.h2.command.query.Query.query (Query.java:475)
    at org.h2.command.CommandContainer.query (CommandContainer.java:251)
    at org.h2.command.Command.executeQuery (Command.java:190)
    at org.h2.jdbc.JdbcPreparedStatement.executeQuery (JdbcPreparedStatement.java:128)
    at org.apache.commons.dbcp2.DelegatingPreparedStatement.executeQuery (DelegatingPreparedStatement.java:123)
    at org.apache.commons.dbcp2.DelegatingPreparedStatement.executeQuery (DelegatingPreparedStatement.java:123)
    at org.owasp.dependencycheck.data.nvdcve.CveDB.updateOrInsertVulnerability (CveDB.java:1396)
    at org.owasp.dependencycheck.data.nvdcve.CveDB.updateVulnerability (CveDB.java:1100)
    at org.owasp.dependencycheck.data.update.nvd.api.NvdApiProcessor.updateCveDb (NvdApiProcessor.java:119)
    at org.owasp.dependencycheck.data.update.nvd.api.NvdApiProcessor.call (NvdApiProcessor.java:96)
    at org.owasp.dependencycheck.data.update.nvd.api.NvdApiProcessor.call (NvdApiProcessor.java:40)
    at java.util.concurrent.FutureTask.run (FutureTask.java:264)
    at java.util.concurrent.ThreadPoolExecutor.runWorker (ThreadPoolExecutor.java:1128)
    at java.util.concurrent.ThreadPoolExecutor$Worker.run (ThreadPoolExecutor.java:628)
    at java.lang.Thread.run (Thread.java:829)
Caused by: java.nio.channels.ClosedChannelException

To Reproduce

<plugin>
                        <groupId>org.owasp</groupId>
                        <artifactId>dependency-check-maven</artifactId>
                        <version>${dependency.check.version}</version>
                        <configuration>
                            <nvdValidForHours>${dependency.nvd.valid.hours}</nvdValidForHours>
                            <skipSystemScope>true</skipSystemScope>
                            <assemblyAnalyzerEnabled>false</assemblyAnalyzerEnabled>
                            <reportSets>
                                <reportSet>
                                    <reports>
                                        <report>aggregate</report>
                                    </reports>
                                </reportSet>
                            </reportSets>
                        </configuration>
                        <executions>
                            <execution>
                                <goals>
                                    <goal>check</goal>
                                </goals>
                            </execution>
                        </executions>
                    </plugin>

Expected behavior
A clear and concise description of what you expected to happen.

Additional context
Add any other context about the problem here.
@sysmat sysmat added the bug label Nov 24, 2024
@jeremylong
Copy link
Owner

Is your disk full? Try purging the database first and recreating it.

@sysmat
Copy link
Author

sysmat commented Nov 25, 2024

@jeremylong no disk is NOT full, what exactly you mean by purging the database first

@nhumblot
Copy link
Collaborator

nhumblot commented Nov 26, 2024 •
edited
Loading

Hi 👋

There is a dependency-check:purge goal which purges the local copy of the NVD data.

@sysmat
Copy link
Author

sysmat commented Nov 26, 2024

@nhumblot thx, ok, I'll try

@sysmat
Copy link
Author

sysmat commented Nov 26, 2024

hm, not working, moving to API we have a lot of problems in CI, and we use a lot of CI resource 12 minutes 24 seconds

@nhumblot
Copy link
Collaborator

nhumblot commented Nov 26, 2024 •
edited
Loading

Just tracking here a quick note here to investigate later when I will have time.

I found a similar behavior on Stack Overflow with H2

The author of the question states:

I had amended an ExecutorService so that if a task took too long it would cancel the task. The task would be cancelled okay but then subsequent database access failed with exceptions

The accepted answer is:

Although not properly documented using the async protocol allows a connection to be interrupted without breaking all other connections.

In H2 database documentation > Pluggable File System:

async: experimental file system that uses AsynchronousFileChannel instead of FileChannel (faster in some operating systems).

To confirm but I will look into the concurrent configuration we use into DependencyCheck and see if I can reproduce an error by relying on some stub that would provoke an interruption 🤔

@sysmat
Copy link
Author

sysmat commented Nov 28, 2024 

[WARNING] Retrying request /rest/json/cves/2.0?resultsPerPage=2000&startIndex=0 : 3 time
[WARNING] NVD API request failures are occurring; retrying request for the 5 time
[INFO] NVD API has 271,677 records in this update
[WARNING] Retrying request /rest/json/cves/2.0?resultsPerPage=2000&startIndex=8000 : 3 time
[WARNING] NVD API request failures are occurring; retrying request for the 5 time
[WARNING] NVD API request failures are occurring; retrying request for the 6 time
[WARNING] NVD API request failures are occurring; retrying request for the 5 time
[WARNING] NVD API request failures are occurring; retrying request for the 5 time
[WARNING] NVD API request failures are occurring; retrying request for the 5 time
[WARNING] NVD API request failures are occurring; retrying request for the 6 time
[WARNING] NVD API request failures are occurring; retrying request for the 6 time
[WARNING] NVD API request failures are occurring; retrying request for the 6 time
[WARNING] NVD API request failures are occurring; retrying request for the 7 time
[WARNING] NVD API request failures are occurring; retrying request for the 7 time
[WARNING] NVD API request failures are occurring; retrying request for the 7 time
[WARNING] NVD API request failures are occurring; retrying request for the 7 time
[WARNING] NVD API request failures are occurring; retrying request for the 8 time
[INFO] Downloaded 10,000/271,677 (4%)
[WARNING] Retrying request /rest/json/cves/2.0?resultsPerPage=2000&startIndex=16000 : 3 time
[WARNING] Retrying request /rest/json/cves/2.0?resultsPerPage=2000&startIndex=22000 : 3 time
[WARNING] NVD API request failures are occurring; retrying request for the 5 time
[INFO] Downloaded 20,000/271,677 (7%)
[WARNING] NVD API request failures are occurring; retrying request for the 6 time
[WARNING] NVD API request failures are occurring; retrying request for the 5 time
[INFO] Downloaded 30,000/271,677 (11%)
[WARNING] Retrying request /rest/json/cves/2.0?resultsPerPage=2000&startIndex=[3000](https://git.arnes.si/docker/maven/-/jobs/1020420#L3000)0 : 3 time
[WARNING] Retrying request /rest/json/cves/2.0?resultsPerPage=2000&startIndex=32000 : 3 time
[WARNING] NVD API request failures are occurring; retrying request for the 5 time
[WARNING] NVD API request failures are occurring; retrying request for the 5 time
[WARNING] NVD API request failures are occurring; retrying request for the 6 time
[WARNING] NVD API request failures are occurring; retrying request for the 5 time
[WARNING] NVD API request failures are occurring; retrying request for the 6 time
[WARNING] NVD API request failures are occurring; retrying request for the 6 time
[WARNING] NVD API request failures are occurring; retrying request for the 7 time
[WARNING] NVD API request failures are occurring; retrying request for the 7 time
[WARNING] NVD API request failures are occurring; retrying request for the 7 time
[WARNING] NVD API request failures are occurring; retrying request for the 8 time
[INFO] Downloaded 40,000/271,677 (15%)
[INFO] Downloaded 50,000/271,677 (18%)
[WARNING] NVD API request failures are occurring; retrying request for the 5 time
[WARNING] Retrying request /rest/json/cves/2.0?resultsPerPage=2000&startIndex=72000 : 3 time
[WARNING] NVD API request failures are occurring; retrying request for the 5 time
[WARNING] NVD API request failures are occurring; retrying request for the 5 time
[WARNING] NVD API request failures are occurring; retrying request for the 6 time
[INFO] Downloaded 60,000/271,677 (22%)
[WARNING] NVD API request failures are occurring; retrying request for the 5 time
[WARNING] NVD API request failures are occurring; retrying request for the 7 time
[WARNING] NVD API request failures are occurring; retrying request for the 5 time
[WARNING] NVD API request failures are occurring; retrying request for the 5 time
[INFO] Downloaded 70,000/271,677 (26%)
[WARNING] Retrying request /rest/json/cves/2.0?resultsPerPage=2000&startIndex=88000 : 3 time
[WARNING] NVD API request failures are occurring; retrying request for the 5 time
[WARNING] NVD API request failures are occurring; retrying request for the 5 time
[WARNING] NVD API request failures are occurring; retrying request for the 6 time
[WARNING] NVD API request failures are occurring; retrying request for the 6 time
[INFO] Downloaded 80,000/271,677 (29%)
[WARNING] NVD API request failures are occurring; retrying request for the 5 time
[WARNING] Retrying request /rest/json/cves/2.0?resultsPerPage=2000&startIndex=62000 : 3 time
[WARNING] NVD API request failures are occurring; retrying request for the 5 time
[WARNING] NVD API request failures are occurring; retrying request for the 5 time
[WARNING] NVD API request failures are occurring; retrying request for the 6 time
[WARNING] Retrying request /rest/json/cves/2.0?resultsPerPage=2000&startIndex=100000 : 3 time
[WARNING] Retrying request /rest/json/cves/2.0?resultsPerPage=2000&startIndex=104000 : 3 time
[WARNING] NVD API request failures are occurring; retrying request for the 5 time
[WARNING] NVD API request failures are occurring; retrying request for the 5 time
[INFO] Downloaded 90,000/271,677 (33%)
[WARNING] Retrying request /rest/json/cves/2.0?resultsPerPage=2000&startIndex=108000 : 3 time
[WARNING] NVD API request failures are occurring; retrying request for the 5 time
[WARNING] NVD API request failures are occurring; retrying request for the 5 time
[WARNING] NVD API request failures are occurring; retrying request for the 5 time
[WARNING] NVD API request failures are occurring; retrying request for the 6 time
[WARNING] NVD API request failures are occurring; retrying request for the 6 time
[WARNING] NVD API request failures are occurring; retrying request for the 7 time
[WARNING] NVD API request failures are occurring; retrying request for the 7 time
[WARNING] NVD API request failures are occurring; retrying request for the 8 time
[WARNING] NVD API request failures are occurring; retrying request for the 5 time
[WARNING] NVD API request failures are occurring; retrying request for the 8 time
[INFO] Downloaded 100,000/271,677 (37%)
  • I have API key, and it runs for 60 minutes 8 seconds, and at end it fails, because CI provider has limit how long cant job run

@nhumblot
Copy link
Collaborator

Hi @sysmat 👋

This length and request failures you are getting in you last log are the same topic as #7178

There are currently issues with the NVD API, which does not belong to the Dependency Check project. You will get more information about this and way to circumvent the problem in this one.

@sysmat
Copy link
Author

sysmat commented Nov 28, 2024

@nhumblot thx for info

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
bug unable to reproduce
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
3 participants
@sysmat @jeremylong @nhumblot