Use url.QueryEscape instead of url.PathEscape to escape query string by pankona · Pull Request #1125 · google/go-github · GitHub
Skip to content

Use url.QueryEscape instead of url.PathEscape to escape query string#1125

Merged
gmlewis merged 6 commits into
google:masterfrom
pankona:1122-fix-search-query-includes-plus
Feb 28, 2019
Merged

Use url.QueryEscape instead of url.PathEscape to escape query string#1125
gmlewis merged 6 commits into
google:masterfrom
pankona:1122-fix-search-query-includes-plus

Conversation

@pankona

@pankona pankona commented Feb 20, 2019

Copy link
Copy Markdown

Fixes #1122.

  • Quit replacing white space with "+"
  • Use url.QueryEscape instead of url.PathEscape to escape query string

@googlebot googlebot added the cla: yes Indication that the PR author has signed a Google Contributor License Agreement. label Feb 20, 2019
@pankona pankona marked this pull request as ready for review February 20, 2019 10:37
@gmlewis

gmlewis commented Feb 20, 2019

Copy link
Copy Markdown
Collaborator

@pankona

pankona commented Feb 20, 2019

Copy link
Copy Markdown
Author

@gmlewis

I've checked #966 and #963 .
I've pushed a new commit 6bc0bca . I think go-querystring has no problem. Could you please check current patch is as you expected?

@gmlewis

gmlewis commented Feb 20, 2019

Copy link
Copy Markdown
Collaborator

Thank you, @pankona! I'll have to take time and look at this tonight... gotta run... but in a nutshell, was #966 just broken? I'm curious how this current PR differs from the original, but don't have time right now to look... I just know there was a problem in #963 that needed fixing and hope that this addresses that issue as well.

@pankona

pankona commented Feb 20, 2019

Copy link
Copy Markdown
Author

@gmlewis

I've checked snippet on #963 (as follows)

rs, _, err := client.Search.Repositories(context.Background(),
		"language:golang+stars:>=200+pushed:>=2018-01-01",
		&github.SearchOptions{Sort: "stars", Order: "desc",
				ListOptions: github.ListOptions{Page: 2, PerPage: 100}})

This query string includes "+" (instead of white space). I think this is NOT acceptable query string.

  • "language:golang+stars:>=200+pushed:>=2018-01-01" doesn't work with GitHub's search input form
  • According to GitHub's search API documentation, there're some examples that include "+" instead of white space, I think the white spaces are already escaped.
  • I think the query string written above should be "language:golang stars:>=200 pushed:>=2018-01-01"

@gmlewis

gmlewis commented Feb 20, 2019

Copy link
Copy Markdown
Collaborator

This query string includes "+" (instead of white space). I think this is NOT acceptable query string.

...although it worked when using curl... does your recommended change also work against the GitHub v3 API? If so, then I'm happy to incorporate your solution.

But are you saying here that the original code was correct?

@pankona

pankona commented Feb 20, 2019

Copy link
Copy Markdown
Author

@gmlewis

But are you saying here that the original code was correct?

Yes. I think original code was correct.

In my understanding, escaping of query string is responsibility of go-github itself, but not of library user. Thus a query string that includes "+" instead of white space may return unexpected result.

curl

  • Here's curl command using "+" instead of white spaces (got OK)
curl -X GET "https://api.github.com/search/repositories?order=desc&page=2&per_page=100&q=language%3Agolang+stars%3A%3E%3D200+pushed%3A%3E%3D2018-01-01&sort=stars"
  • Here's curl command using white spaces directly (got error)
curl -X GET "https://api.github.com/search/repositories?order=desc&page=2&per_page=100&q=language%3Agolang stars%3A%3E%3D200 pushed%3A%3E%3D2018-01-01&sort=stars"

In case of using curl, escaping of query string is responsibility of user. Thus former case goes okay.

@gmlewis

gmlewis commented Feb 20, 2019

Copy link
Copy Markdown
Collaborator

Thank you for the analysis, @pankona! It is greatly appreciated!

@gauntface - do you agree?

TL;DR: #966 was the wrong fix for #963 - the query in #963 was malformed. This PR goes back to the original implementation.

@gmlewis gmlewis requested a review from gauntface February 20, 2019 14:12
@gauntface

Copy link
Copy Markdown
Contributor

Ok, my understanding from everything is that this is the correct change.

Forcing users to not do this:

client.Search.Repositories(ctx, "language:golang+stars:>=200+pushed:>=2018-01-01", ...)

And instead do this:

client.Search.Repositories(ctx, "language:golang stars:>=200 pushed:>=2018-01-01", ...)

I like this approach just from the perspective of being close to the input you would provide to the Github.com search box.

Regarding this PR, @pankona could you add a comment to the search() method calling out the example input of " " vs "+" and highlight that we expect " ".

@pankona

pankona commented Feb 20, 2019

Copy link
Copy Markdown
Author

@gauntface
Thank you for checking!

comment

okay, I will.

@gmlewis gmlewis left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

One tiny nit, then LGTM.
Sorry I missed that before.
Thank you, @pankona!

Awaiting second LGTM before merge.

Comment thread github/search.go Outdated
// Helper function that executes search queries against different
// GitHub search types (repositories, commits, code, issues, users, labels)
//
// If *searchParameters.Query includes multiple condition, it MUST NOT include "+" as condition separator.

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit: please remove leading * as we typically do not put these for pointers in Godocs.

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks! fixed on 63ab8d9

@gmlewis gmlewis left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Great! Thank you, @pankona!

One more quick request since @gauntface hasn't reviewed yet, please...

Could you please add language:c++ to line 118 in search_test.go just to make your comment super-clear in the test results for how it will look?

@pankona

pankona commented Feb 22, 2019

Copy link
Copy Markdown
Author

@gmlewis

understood, I will!

@pankona

pankona commented Feb 22, 2019

Copy link
Copy Markdown
Author

@gmlewis

At b94a2b5, I introduced c++ in unit tests to make escape result more clear. How about it?

@gmlewis

gmlewis commented Feb 22, 2019

Copy link
Copy Markdown
Collaborator

Perfect, @pankona ! Thank you!

Awaiting @gauntface LGTM before merging.

@pankona

pankona commented Feb 27, 2019

Copy link
Copy Markdown
Author

Gently ping to @gauntface :)

@gauntface gauntface left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sorry for the delayed LGTM. This looks great. Thank you so much for the change 🎉

@pankona

pankona commented Feb 28, 2019

Copy link
Copy Markdown
Author

Thank you @gauntface for your confirmation!

@gmlewis

If there's no concern for merging, could you please proceed?

@gmlewis

gmlewis commented Feb 28, 2019

Copy link
Copy Markdown
Collaborator

@gmlewis gmlewis merged commit 7462feb into google:master Feb 28, 2019
@pankona pankona deleted the 1122-fix-search-query-includes-plus branch February 28, 2019 02:57
n1lesh pushed a commit to n1lesh/go-github that referenced this pull request Oct 2, 2020
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

cla: yes Indication that the PR author has signed a Google Contributor License Agreement.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants