{{ message }}
use urlopen context manager to handle connection close in a timely manner#47
Closed
jubrad wants to merge 1 commit into
Closed
use urlopen context manager to handle connection close in a timely manner#47jubrad wants to merge 1 commit into
jubrad wants to merge 1 commit into
Conversation
jubrad
force-pushed
the
close-connection
branch
from
e1055bd to
bb3a0c7
Compare
Owner
|
This fix looks good, resulting in |
|
Seems good to me |
kevinsteves
added a commit
that referenced
this pull request
Jan 13, 2023
#47 from Justin Bradfield. 1. Use urlopen() as a context manager 2. Immediately read() response 3. Set object 'pan_body' attribute to read() result
Owner
|
Modified version of |
Author
@kevinsteves I think I did this to log a few things during debug. I then removed the log messages and didn't put the code back to the simpler form. Thanks for merging. |
Author
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
3 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What it does:
More explicitly closes the xapi response object to avoid downstream circular reference or other potential issues where garbage collector does not promptly clean the request and close the connection.
Background:
I'm noticing many "Session Timed Out" errors while using local-execs in terraform to configure some resources that the tf provider cannot. At first I tested to see what rate I would hit this bug, but was not able to with >6k requests happening serially (creating and deleting the same dns proxy static entries). Since terraform runs with 10 parallel threads walking its tree I decided to test some parallel calls and quickly was able to rerpro. ~20-40 threads making 4 entries each.
I believe this relates to terraform-provider-panos/issues/255, at least it does for me, but it's possible there's something wonky going on with the PA server here.
How it was tested:
Using a python script that adds and removes dns-proxy static entries I validated that with ~20 processes each making 8 entries as fast as they can the python client would regularly produce "Session Timed Out" errors. My goal of this test was to see if I could increase the rate of collision or push the server beyond its concurrent open request limits. These "errors" would produce a valid http response to the urlopen response object with the following error:
b'<response status="unauth" code="22"><msg><line>Session timed out</line></msg></response>'To me, this indicates that there's something weird going on on the server side, potentially to do with auth, but probably not due to any tcp / request issues.
When using the context manager I could not reproduce this error in python, even when increasing the concurrent processes to 40. Although... the terraform provider does give me a
session timed outso there's still more for me to figure out.Not the best solution, but it's what I have time for.