{{ message }}
Repository navigation
Unable to create database for Maven project #89
Description
An error occurred when I used the CLI program to create a database for maven project:
❯ codeql database create java-database --language=java --command='mvn clean install'
Initializing database at /Users/jin/Desktop/micro_service_seclab/java-database.
Running build command: [mvn, clean, install]
[2021-11-03 22:03:17] [ERROR] Spawned process exited abnormally (code 137; tried to run: [/Users/jin/codeql/tools/osx64/preload_tracer, mvn, clean, install])
A fatal error occurred: Exit status 137 from command: [mvn, clean, install]
The following is the log information generated:
...
[2021-11-03 22:05:13] [DETAILS] database init> Found candidate extractor root for java: /Users/jin/codeql/java.
[2021-11-03 22:05:13] [PROGRESS] database init> Successfully loaded extractor Java (java) from /Users/jin/codeql/java.
[2021-11-03 22:05:13] [PROGRESS] database init> Created skeleton CodeQL database at /Users/jin/Desktop/micro_service_seclab/java-database. This in-progress database is ready to be populated by an extractor.
[2021-11-03 22:05:13] Plumbing command codeql database init completed.
[2021-11-03 22:05:13] [PROGRESS] database create> Running build command: [mvn, clean, install]
[2021-11-03 22:05:13] Running plumbing command: codeql database trace-command --working-dir=/Users/jin/Desktop/micro_service_seclab --index-traceless-dbs --no-db-cluster -- /Users/jin/Desktop/micro_service_seclab/java-database mvn clean install
[2021-11-03 22:05:13] Setting up tracing from specification at /Users/jin/codeql/java/tools/compiler-tracing.spec.
[2021-11-03 22:05:13] Picked up extra JVM args: '-javaagent:/Users/jin/codeql/java/tools/codeql-java-agent.jar=ignore-project,java' '-Xbootclasspath/a:/Users/jin/codeql/java/tools/codeql-java-agent.jar'
[2021-11-03 22:05:13] [PROGRESS] database trace-command> Running command in /Users/jin/Desktop/micro_service_seclab: [mvn, clean, install]
[2021-11-03 22:05:16] [ERROR] Spawned process exited abnormally (code 137; tried to run: [/Users/jin/codeql/tools/osx64/preload_tracer, mvn, clean, install])
[2021-11-03 22:05:16] Exception caught at top level: Exit status 137 from command: [mvn, clean, install]
com.semmle.cli2.database.DatabaseProcessCommandCommon.executeSubcommand(DatabaseProcessCommandCommon.java:188)
com.semmle.cli2.database.TraceCommandCommand.executeSubcommand(TraceCommandCommand.java:86)
com.semmle.cli2.picocli.PlumbingRunner.run(PlumbingRunner.java:110)
com.semmle.cli2.picocli.SubcommandCommon.runPlumbingInProcess(SubcommandCommon.java:160)
com.semmle.cli2.database.CreateCommand.executeSubcommand(CreateCommand.java:151)
com.semmle.cli2.picocli.SubcommandCommon.call(SubcommandCommon.java:456)
com.semmle.cli2.picocli.SubcommandMaker.runMain(SubcommandMaker.java:205)
com.semmle.cli2.picocli.SubcommandMaker.runMain(SubcommandMaker.java:214)
com.semmle.cli2.CodeQL.main(CodeQL.java:98)
The environment variable 'mvn' has been added and is working normally.
platform: macos 12 Monterey / m1 arm64
Activity
hmakholm commentedon Nov 3, 2021
Unfortunately, database creation on M1-based Macintoshes does not currently work reliably.
We're actively working on the matter, but it's going to take some time yet to resolve some fascinating interactions between how CodeQL instruments the build environment to observe what is being compiled, and how MacOS supports mix-and-matching between binaries compiled for Intel and ones compiled natively for M1.
fade03 commentedon Nov 3, 2021
Thank you for showing me what the problem is, creation of databases in other languages (such as Python) is working :-)
hmakholm commentedon Nov 3, 2021
Yes -- the difference is that for Python there's no "build process" we can attempt to observe, so CodeQL will just gobble up all the
.pyfiles it finds in the source tree and assume they're part of the program. With Java we can (usually) do better than that, but a side effect of that is that we have no "take all of the.javafiles" workflow ready to fall back on when the better solution fails, such as on M1 macs for the time being.