src: make parsing compatible with motdotla/dotenv package by marekpiechut · Pull Request #54215 · nodejs/node · GitHub
Skip to content

src: make parsing compatible with motdotla/dotenv package#54215

Open
marekpiechut wants to merge 3 commits intonodejs:mainfrom
marekpiechut:env-inner-quotes
Open

src: make parsing compatible with motdotla/dotenv package#54215
marekpiechut wants to merge 3 commits intonodejs:mainfrom
marekpiechut:env-inner-quotes

Conversation

@marekpiechut
Copy link
Copy Markdown
Contributor

Fixes: #54134

@nodejs-github-bot
Copy link
Copy Markdown
Collaborator

@nodejs-github-bot nodejs-github-bot added build Issues and PRs related to build files or the CI. c++ Issues and PRs that require attention from people who are familiar with C++. needs-ci PRs that need a full CI run. labels Aug 5, 2024
@marekpiechut
Copy link
Copy Markdown
Contributor Author

Hmm, no idea which subsystem to use here 🤔. I see that previous commits in these files were marked as src. Should I also use that?

Comment thread src/node_dotenv.cc Outdated
auto first = input.front();
if ((first == '\'' || first == '"' || first == '`') &&
input.back() == first) {
input = input.substr(1, input.size() - 2);
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

A more readable way:

input.remove_prefix(1);
input.remove_suffix(2);

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed

Comment thread src/node_dotenv.cc
Comment thread src/node_dotenv.cc

void Dotenv::ParseContent(const std::string_view input) {
std::string lines(input);
std::string_view parse_key(std::string_view key) {
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can you add a comment to what this function does?

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done

Comment thread src/node_dotenv.cc Outdated
// Expand \n to newline in double-quote strings
size_t pos = 0;
auto expanded = std::string(trimmed);
while ((pos = expanded.find("\\n", pos)) != std::string_view::npos) {
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

expanded is a string now. std::string::npos is the correct return value here.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

👍Done, thanks

Comment thread src/node_dotenv.cc Outdated
if (value.empty()) return "";

auto trimmed = trim_quotes(value);
if (value.front() == '\"' && value.back() == '\"') {
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can you add a documentation to here?
Returning the else statement early will make this function much more readable.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done

Comment thread src/node_dotenv.cc
std::string::size_type start = 0;
std::string::size_type end = 0;

for (std::string::size_type i = 0; i < input.size(); i++) {
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You removed all of the comments in this function, and it is a lot less readable right now. I can't review it with the current state.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is basically a new function. Old one was doing a lot of back & forth searching for chars and newlines in string and it was hard for me to fit in a fix. This goes through the content of file only once and parses it char by char.

Comment thread src/node_dotenv.cc Outdated
}
}

std::optional<std::string> Dotenv::GetValue(const std::string_view key) const {
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This can return std::string_view since the underlying store will not be disposed.

Suggested change
std::optional<std::string> Dotenv::GetValue(const std::string_view key) const {
std::optional<std::string_view> Dotenv::GetValue(const std::string_view key) const {

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done

Comment thread src/node_dotenv.cc Outdated
auto match = store_.find(key.data());

if (match != store_.end()) {
return std::optional<std::string>{match->second};
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
return std::optional<std::string>{match->second};
return match->second;

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done

Comment thread test/parallel/test-dotenv.js Outdated
Comment on lines +75 to +79

// These are no longer true, parser is now more strict when it comes to balancing double quotes
// assert.strictEqual(process.env.MULTI_NOT_VALID_QUOTE, '"');
// assert.strictEqual(process.env.MULTI_NOT_VALID, 'THIS');

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
// These are no longer true, parser is now more strict when it comes to balancing double quotes
// assert.strictEqual(process.env.MULTI_NOT_VALID_QUOTE, '"');
// assert.strictEqual(process.env.MULTI_NOT_VALID, 'THIS');

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done

@marekpiechut marekpiechut changed the title dotenv: make parsing compatible with motdotla/dotenv package src: make parsing compatible with motdotla/dotenv package Aug 5, 2024
@avivkeller avivkeller added the dotenv Issues and PRs related to .env file parsing label Aug 5, 2024
@codecov
Copy link
Copy Markdown

codecov Bot commented Aug 5, 2024

Codecov Report

Attention: Patch coverage is 96.20253% with 3 lines in your changes missing coverage. Please review.

Project coverage is 87.09%. Comparing base (67f7137) to head (00fa796).
Report is 634 commits behind head on main.

Files with missing lines Patch % Lines
src/node_dotenv.cc 96.20% 0 Missing and 3 partials ⚠️
Additional details and impacted files
@@           Coverage Diff           @@
##             main   #54215   +/-   ##
=======================================
  Coverage   87.09%   87.09%           
=======================================
  Files         647      647           
  Lines      181845   181900   +55     
  Branches    34913    34924   +11     
=======================================
+ Hits       158373   158426   +53     
+ Misses      16751    16749    -2     
- Partials     6721     6725    +4     
Files with missing lines Coverage Δ
src/node_dotenv.h 100.00% <ø> (ø)
src/node_dotenv.cc 84.35% <96.20%> (+3.98%) ⬆️

... and 27 files with indirect coverage changes

@marekpiechut
Copy link
Copy Markdown
Contributor Author

Hey @anonrig did you have a chance to take a second look? I've resolved all small things and only thing left is the parsing function that I have rewritten.

If you think that the change is too risky I can revert it, keep the tests and try to fit in the fix in current implementation.

@tniessen
Copy link
Copy Markdown
Member

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

build Issues and PRs related to build files or the CI. c++ Issues and PRs that require attention from people who are familiar with C++. dotenv Issues and PRs related to .env file parsing needs-ci PRs that need a full CI run.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

--env-file does not support inner quotes (does not behave like dotenv)

5 participants