Swift: skip QL code generation on untouched files · github/codeql@2cd5881 · GitHub
Skip to content

Commit 2cd5881

Browse files
committed
Swift: skip QL code generation on untouched files
This is a developer QoL improvement, where running codegen will skip writing (and especially formatting) any files that were not changed. **Why?** While code generation in itself was pretty much instant, QL formatting of generated code was starting to take a long time. This made unconditionally running codegen quite annoying, for example before each test run as part of an IDE workflow or as part of the pre-commit hook. **How?** This was not completely straightforward as we could not work with the contents of the file prior to code generation as that was already post-processed by the QL formatting, so we had no chance of comparing the output of template rendering with that. We therefore store the hashes of the files _prior_ to QL formatting in a checked-in file (`swift/ql/.generated.list`). We can therefore load those hashes at the beginning of code generation, use them to compare the template rendering output and update them in this special registry file. **What else?** We also extend this mechanism to detect accidental modification of generated files in a more robust way. Before this patch, we were doing it with a rough regexp based heuristic. Now, we just store the hashes of the files _after_ QL formatting in the same checked file, so we can check that and stop generation if a generated file was modified, or a stub was modified without removing the `// generated` header.
1 parent 0796926 commit 2cd5881

9 files changed

Lines changed: 1323 additions & 209 deletions

File tree

‎.github/workflows/swift.yml‎

Lines changed: 1 addition & 0 deletions

‎.pre-commit-config.yaml‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -44,7 +44,7 @@ repos:
4444

4545
- id: swift-codegen
4646
name: Run Swift checked in code generation
47-
files: ^swift/(schema.py$|codegen/|.*/generated/|ql/lib/(swift\.dbscheme$|codeql/swift/elements))
47+
files: ^swift/(schema.py$|codegen/|.*/generated/|ql/lib/(swift\.dbscheme$|codeql/swift/elements)|ql/\.generated.list)
4848
language: system
4949
entry: bazel run //swift/codegen -- --quiet
5050
pass_filenames: false

‎swift/codegen/codegen.py‎

Lines changed: 5 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -40,11 +40,13 @@ def _parse_args() -> argparse.Namespace:
4040
p.add_argument("--codeql-binary", default="codeql", help="command to use for QL formatting (default %(default)s)")
4141
p.add_argument("--cpp-output", type=_abspath,
4242
help="output directory for generated C++ files, required if trap or cpp is provided to --generate")
43+
p.add_argument("--generated-registry", type=_abspath, default=paths.swift_dir / "ql/.generated.list",
44+
help="registry file containing information about checked-in generated code")
4345
return p.parse_args()
4446

4547

46-
def _abspath(x: str) -> pathlib.Path:
47-
return pathlib.Path(x).resolve()
48+
def _abspath(x: str) -> typing.Optional[pathlib.Path]:
49+
return pathlib.Path(x).resolve() if x else None
4850

4951

5052
def run():
@@ -56,9 +58,8 @@ def run():
5658
else:
5759
log_level = logging.INFO
5860
logging.basicConfig(format="{levelname} {message}", style='{', level=log_level)
59-
exe_path = paths.exe_file.relative_to(opts.swift_dir)
6061
for target in opts.generate:
61-
generate(target, opts, render.Renderer(exe_path))
62+
generate(target, opts, render.Renderer(opts.swift_dir))
6263

6364

6465
if __name__ == "__main__":

‎swift/codegen/generators/qlgen.py‎

Lines changed: 83 additions & 103 deletions

0 commit comments

Comments
 (0)