Manage Lambda code in code objects / share code between identical versions by dfangl · Pull Request #7019 · localstack/localstack · GitHub
Skip to content
This repository was archived by the owner on Mar 23, 2026. It is now read-only.

Manage Lambda code in code objects / share code between identical versions#7019

Merged
dfangl merged 12 commits into
masterfrom
lambda-code-rework
Oct 19, 2022
Merged

Manage Lambda code in code objects / share code between identical versions#7019
dfangl merged 12 commits into
masterfrom
lambda-code-rework

Conversation

@dfangl

@dfangl dfangl commented Oct 13, 2022

Copy link
Copy Markdown
Member

Motivation

Currently, we store the code of a lambda on disk for every version. This has the following problems:

  • inefficiency, mainly needing to unzip for every publish as well as every update
  • since unzipping can take a while, the lambda version state will not be active on first return

Changes

This PR fixes this behavior by encapsulating the unzipped code on disk with the code storage in S3. As long as the code is not updated, it will be shared (across versions of a function, never across functions).
This leads to faster preparation times and better API parity.

The publish version preparation is now nearly instantaneous, and therefore handled synchronously. The only exception is when publish is triggered by a state changing operation, like the publish=True parameter of create_function and update_function_code, in this case it will still be asynchronous, and both versions will wait for the unzipping (but its only unzipped once, due to a lock).

Cleanups will be only done after ensuring no other version is taking up the code, on delete of the version or on function updates. Those complete cleanups will also delete the data in S3.
Cache cleanups of the code, namely cleaning up only the unzipped code on disk, will be done on service stop. In this case, the S3 data is not deleted, to enable persistence.

@dfangl dfangl temporarily deployed to localstack-ext-tests October 13, 2022 13:55 Inactive
@github-actions

github-actions Bot commented Oct 13, 2022

Copy link
Copy Markdown

@dfangl dfangl force-pushed the lambda-code-rework branch from 7af1a92 to e7f47d3 Compare October 13, 2022 16:06
@dfangl dfangl temporarily deployed to localstack-ext-tests October 13, 2022 16:06 Inactive
@dfangl dfangl temporarily deployed to localstack-ext-tests October 13, 2022 19:05 Inactive
@dfangl dfangl force-pushed the lambda-code-rework branch from db1c508 to 9439d7c Compare October 14, 2022 15:04
@dfangl dfangl temporarily deployed to localstack-ext-tests October 14, 2022 15:04 Inactive
@coveralls

coveralls commented Oct 14, 2022

Copy link
Copy Markdown

@dfangl dfangl temporarily deployed to localstack-ext-tests October 18, 2022 09:24 Inactive
@dfangl dfangl marked this pull request as ready for review October 18, 2022 09:24
@dfangl dfangl temporarily deployed to localstack-ext-tests October 18, 2022 09:34 Inactive
@dfangl dfangl temporarily deployed to localstack-ext-tests October 18, 2022 09:36 Inactive

@dominikschubert dominikschubert left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Didn't see any major issues 👍

Really awesome that we can now properly use versions "instantly" without skipping all the state verifications 😎 🚀

Comment thread tests/integration/awslambda/test_lambda_api.py
Comment thread localstack/services/awslambda/invocation/lambda_service.py Outdated
Comment thread localstack/services/awslambda/provider.py
Comment thread localstack/services/awslambda/invocation/lambda_service.py
Comment thread localstack/services/awslambda/invocation/lambda_models.py
@dfangl dfangl force-pushed the lambda-code-rework branch from 050c1cb to c47238b Compare October 18, 2022 14:41
@dfangl dfangl temporarily deployed to localstack-ext-tests October 18, 2022 14:42 Inactive
@dfangl dfangl temporarily deployed to localstack-ext-tests October 18, 2022 15:04 Inactive
@dfangl dfangl temporarily deployed to localstack-ext-tests October 18, 2022 15:05 Inactive
Comment thread localstack/services/awslambda/invocation/lambda_service.py Outdated
Co-authored-by: Dominik Schubert <dominik.schubert91@gmail.com>
@dfangl dfangl temporarily deployed to localstack-ext-tests October 19, 2022 07:29 Inactive
@dfangl dfangl merged commit c19e69c into master Oct 19, 2022
@dfangl dfangl deleted the lambda-code-rework branch October 19, 2022 07:29
cmoralesmx pushed a commit to cmoralesmx/localstack that referenced this pull request Oct 24, 2022
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants