Skip to content

Commit 8595105

Browse files
raulcdraulcd
authored
GH-49743: [Release] Split the vote thread preparation into its own shell script (#49770)
### Rationale for this change Currently running the script is failing if default flags are used and also failing if we run the source vote in separation as documented on the release.rst page. ### What changes are included in this PR? Splitting the script into two each one for it's own purpose will make it easier to focus and have less "flag" incompatibilities which to be fair is unnecessary for the purpose of these minor scripts. ### Are these changes tested? Yes, I've tested the new script in isolation to generate a new vote email. ### Are there any user-facing changes? No * GitHub Issue: #49743 Lead-authored-by: Sutou Kouhei <kou@clear-code.com> Co-authored-by: Raúl Cumplido <raulcumplido@gmail.com> Signed-off-by: Raúl Cumplido <raulcumplido@gmail.com>
1 parent f092b91 commit 8595105

5 files changed

Lines changed: 270 additions & 155 deletions

File tree

‎dev/release/02-source-test.rb‎

Lines changed: 0 additions & 83 deletions
Original file line numberDiff line numberDiff line change
@@ -46,8 +46,6 @@ def source(*targets)
4646
sh(env, @tarball_script, @release_version, "0")
4747
FileUtils.mkdir_p("artifacts")
4848
sh("mv", @archive_name, "artifacts/")
49-
File.write("artifacts/#{@archive_name}.sha512",
50-
sh(env, "shasum", "-a", "512", "artifacts/#{@archive_name}"))
5149
output = sh(env, @script, @release_version, "0")
5250
sh("tar", "xf", "artifacts/#{@archive_name}")
5351
output
@@ -74,85 +72,4 @@ def test_python_version
7472
Dir.glob("dist/pyarrow-*.tar.gz"))
7573
end
7674
end
77-
78-
def test_vote
79-
github_token = File.read(@env)[/^GH_TOKEN=(.*)$/, 1]
80-
uri = URI.parse("https://api.github.com/graphql")
81-
n_issues_query = {
82-
"query" => <<-QUERY,
83-
query {
84-
search(query: "repo:apache/arrow is:issue is:closed milestone:#{@release_version}",
85-
type: ISSUE) {
86-
issueCount
87-
}
88-
}
89-
QUERY
90-
}
91-
response = Net::HTTP.post(uri,
92-
n_issues_query.to_json,
93-
"Content-Type" => "application/json",
94-
"Authorization" => "Bearer #{github_token}")
95-
n_resolved_issues = JSON.parse(response.body)["data"]["search"]["issueCount"]
96-
github_repository = ENV["GITHUB_REPOSITORY"] || "apache/arrow"
97-
github_api_url = "https://api.github.com"
98-
verify_prs = URI("#{github_api_url}/repos/#{github_repository}/pulls" +
99-
"?state=open" +
100-
"&head=apache:release-#{@release_version}-rc0")
101-
verify_pr_url = nil
102-
headers = {
103-
"Accept" => "application/vnd.github+json",
104-
}
105-
106-
if github_token
107-
headers["Authorization"] = "Bearer #{github_token}"
108-
end
109-
verify_prs.open(headers) do |response|
110-
verify_pr_url = (JSON.parse(response.read)[0] || {})["html_url"]
111-
end
112-
output = source("VOTE")
113-
tarball_hash = Digest::SHA512.file("artifacts/#{@archive_name}").to_s
114-
assert_equal(<<-VOTE.strip, output[/^-+$(.+?)^-+$/m, 1].strip)
115-
To: dev@arrow.apache.org
116-
Subject: [VOTE] Release Apache Arrow #{@release_version} - RC0
117-
118-
Hi,
119-
120-
I would like to propose the following release candidate (RC0) of Apache
121-
Arrow version #{@release_version}. This is a release consisting of #{n_resolved_issues}
122-
resolved GitHub issues[1].
123-
124-
This release candidate is based on commit:
125-
#{@current_commit} [2]
126-
127-
The source release rc0 is hosted at [3].
128-
The binary artifacts are hosted at [4][5][6][7][8][9].
129-
The changelog is located at [10].
130-
131-
Please download, verify checksums and signatures, run the unit tests,
132-
and vote on the release. See [11] for the SHA-512 checksum for this RC and [12]
133-
for how to validate a release candidate.
134-
135-
See also a verification result on GitHub pull request [13].
136-
137-
The vote will be open for at least 72 hours.
138-
139-
[ ] +1 Release this as Apache Arrow #{@release_version}
140-
[ ] +0
141-
[ ] -1 Do not release this as Apache Arrow #{@release_version} because...
142-
143-
[1]: https://github.com/apache/arrow/issues?q=is%3Aissue+milestone%3A#{@release_version}+is%3Aclosed
144-
[2]: https://github.com/apache/arrow/tree/#{@current_commit}
145-
[3]: https://dist.apache.org/repos/dist/dev/arrow/apache-arrow-#{@release_version}-rc0
146-
[4]: https://packages.apache.org/artifactory/arrow/almalinux-rc/
147-
[5]: https://packages.apache.org/artifactory/arrow/amazon-linux-rc/
148-
[6]: https://packages.apache.org/artifactory/arrow/centos-rc/
149-
[7]: https://packages.apache.org/artifactory/arrow/debian-rc/
150-
[8]: https://packages.apache.org/artifactory/arrow/ubuntu-rc/
151-
[9]: https://github.com/apache/arrow/releases/tag/apache-arrow-#{@release_version}-rc0
152-
[10]: https://github.com/apache/arrow/blob/#{@current_commit}/CHANGELOG.md
153-
[11]: #{tarball_hash}
154-
[12]: https://arrow.apache.org/docs/developers/release_verification.html
155-
[13]: #{verify_pr_url || "null"}
156-
VOTE
157-
end
15875
end

‎dev/release/02-source.sh‎

Lines changed: 0 additions & 71 deletions
Original file line numberDiff line numberDiff line change
@@ -25,7 +25,6 @@ set -eu
2525
: ${SOURCE_RAT:=${SOURCE_DEFAULT}}
2626
: ${SOURCE_UPLOAD:=${SOURCE_DEFAULT}}
2727
: ${SOURCE_PR:=${SOURCE_DEFAULT}}
28-
: ${SOURCE_VOTE:=${SOURCE_DEFAULT}}
2928

3029
SOURCE_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
3130
SOURCE_TOP_DIR="$(cd "${SOURCE_DIR}/../../" && pwd)"
@@ -114,73 +113,3 @@ if [ ${SOURCE_PR} -gt 0 ]; then
114113
--verify-source \
115114
--version=${version}
116115
fi
117-
118-
if [ ${SOURCE_VOTE} -gt 0 ]; then
119-
curl_common_options=(--header "Authorization: Bearer ${GH_TOKEN}")
120-
121-
curl_options=("${curl_common_options[@]}")
122-
curl_options+=(--data "{\"query\": \"query {search(query: \\\"repo:apache/arrow is:issue is:closed milestone:${version}\\\", type:ISSUE) {issueCount}}\"}")
123-
curl_options+=("https://api.github.com/graphql")
124-
n_resolved_issues=$(curl "${curl_options[@]}" | jq ".data.search.issueCount")
125-
126-
curl_options=("${curl_common_options[@]}")
127-
curl_options+=(--header "Accept: application/vnd.github+json")
128-
curl_options+=(--get)
129-
curl_options+=(--data "state=open")
130-
curl_options+=(--data "head=apache:${rc_branch}")
131-
curl_options+=(https://api.github.com/repos/${GITHUB_REPOSITORY}/pulls)
132-
verify_pr_url=$(curl "${curl_options[@]}" | jq -r ".[0].html_url")
133-
# Read the checksum so we can include it in the vote thread email.
134-
sha512_path="artifacts/${tarball}.sha512"
135-
[[ -f "${sha512_path}" ]] || { echo "Error: ${sha512_path} must exist"; exit 1; }
136-
tarball_hash=$(cat "${sha512_path}" | awk '{print $1}')
137-
138-
echo "The following draft email has been created to send to the"
139-
echo "dev@arrow.apache.org mailing list"
140-
echo ""
141-
echo "---------------------------------------------------------"
142-
cat <<MAIL
143-
To: dev@arrow.apache.org
144-
Subject: [VOTE] Release Apache Arrow ${version} - RC${rc}
145-
146-
Hi,
147-
148-
I would like to propose the following release candidate (RC${rc}) of Apache
149-
Arrow version ${version}. This is a release consisting of ${n_resolved_issues}
150-
resolved GitHub issues[1].
151-
152-
This release candidate is based on commit:
153-
${release_hash} [2]
154-
155-
The source release rc${rc} is hosted at [3].
156-
The binary artifacts are hosted at [4][5][6][7][8][9].
157-
The changelog is located at [10].
158-
159-
Please download, verify checksums and signatures, run the unit tests,
160-
and vote on the release. See [11] for the SHA-512 checksum for this RC and [12]
161-
for how to validate a release candidate.
162-
163-
See also a verification result on GitHub pull request [13].
164-
165-
The vote will be open for at least 72 hours.
166-
167-
[ ] +1 Release this as Apache Arrow ${version}
168-
[ ] +0
169-
[ ] -1 Do not release this as Apache Arrow ${version} because...
170-
171-
[1]: https://github.com/apache/arrow/issues?q=is%3Aissue+milestone%3A${version}+is%3Aclosed
172-
[2]: https://github.com/apache/arrow/tree/${release_hash}
173-
[3]: ${rc_url}
174-
[4]: https://packages.apache.org/artifactory/arrow/almalinux-rc/
175-
[5]: https://packages.apache.org/artifactory/arrow/amazon-linux-rc/
176-
[6]: https://packages.apache.org/artifactory/arrow/centos-rc/
177-
[7]: https://packages.apache.org/artifactory/arrow/debian-rc/
178-
[8]: https://packages.apache.org/artifactory/arrow/ubuntu-rc/
179-
[9]: https://github.com/apache/arrow/releases/tag/apache-arrow-${version}-rc${rc}
180-
[10]: https://github.com/apache/arrow/blob/${release_hash}/CHANGELOG.md
181-
[11]: ${tarball_hash}
182-
[12]: https://arrow.apache.org/docs/developers/release_verification.html
183-
[13]: ${verify_pr_url}
184-
MAIL
185-
echo "---------------------------------------------------------"
186-
fi

‎dev/release/09-vote-email-test.rb‎

Lines changed: 136 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,136 @@
1+
# Licensed to the Apache Software Foundation (ASF) under one
2+
# or more contributor license agreements. See the NOTICE file
3+
# distributed with this work for additional information
4+
# regarding copyright ownership. The ASF licenses this file
5+
# to you under the Apache License, Version 2.0 (the
6+
# "License"); you may not use this file except in compliance
7+
# with the License. You may obtain a copy of the License at
8+
#
9+
# http://www.apache.org/licenses/LICENSE-2.0
10+
#
11+
# Unless required by applicable law or agreed to in writing,
12+
# software distributed under the License is distributed on an
13+
# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
14+
# KIND, either express or implied. See the License for the
15+
# specific language governing permissions and limitations
16+
# under the License.
17+
18+
class VoteEmailTest < Test::Unit::TestCase
19+
include GitRunnable
20+
include VersionDetectable
21+
22+
def setup
23+
@current_commit = git_current_commit
24+
detect_versions
25+
@tag_name_no_rc = "apache-arrow-#{@release_version}"
26+
@archive_name = "apache-arrow-#{@release_version}.tar.gz"
27+
@script = File.expand_path("dev/release/09-vote-email.sh")
28+
@tarball_script = File.expand_path("dev/release/utils-create-release-tarball.sh")
29+
@env = File.expand_path("dev/release/.env")
30+
31+
Dir.mktmpdir do |dir|
32+
Dir.chdir(dir) do
33+
yield
34+
end
35+
end
36+
end
37+
38+
def vote_email(*targets)
39+
env = {
40+
"VOTE_DEFAULT" => "0",
41+
"release_hash" => @current_commit,
42+
}
43+
targets.each do |target|
44+
env["VOTE_#{target}"] = "1"
45+
end
46+
sh(env, @tarball_script, @release_version, "0")
47+
FileUtils.mkdir_p("artifacts")
48+
sh("mv", @archive_name, "artifacts/")
49+
Dir.chdir("artifacts") do
50+
File.write("#{@archive_name}.sha512",
51+
sh(env, "shasum", "-a", "512", @archive_name))
52+
end
53+
sh(env, @script, @release_version, "0")
54+
end
55+
56+
def test_template
57+
github_token = File.read(@env)[/^GH_TOKEN=(.*)$/, 1]
58+
uri = URI.parse("https://api.github.com/graphql")
59+
n_issues_query = {
60+
"query" => <<-QUERY,
61+
query {
62+
search(query: "repo:apache/arrow is:issue is:closed milestone:#{@release_version}",
63+
type: ISSUE) {
64+
issueCount
65+
}
66+
}
67+
QUERY
68+
}
69+
response = Net::HTTP.post(uri,
70+
n_issues_query.to_json,
71+
"Content-Type" => "application/json",
72+
"Authorization" => "Bearer #{github_token}")
73+
n_resolved_issues = JSON.parse(response.body)["data"]["search"]["issueCount"]
74+
github_repository = ENV["GITHUB_REPOSITORY"] || "apache/arrow"
75+
github_api_url = "https://api.github.com"
76+
verify_prs = URI("#{github_api_url}/repos/#{github_repository}/pulls" +
77+
"?state=open" +
78+
"&head=apache:release-#{@release_version}-rc0")
79+
verify_pr_url = nil
80+
headers = {
81+
"Accept" => "application/vnd.github+json",
82+
}
83+
84+
if github_token
85+
headers["Authorization"] = "Bearer #{github_token}"
86+
end
87+
verify_prs.open(headers) do |response|
88+
verify_pr_url = (JSON.parse(response.read)[0] || {})["html_url"]
89+
end
90+
output = vote_email("TEMPLATE")
91+
tarball_hash = Digest::SHA512.file("artifacts/#{@archive_name}").to_s
92+
assert_equal(<<-TEMPLATE.strip, output[/^-+$(.+?)^-+$/m, 1].strip)
93+
To: dev@arrow.apache.org
94+
Subject: [VOTE] Release Apache Arrow #{@release_version} - RC0
95+
96+
Hi,
97+
98+
I would like to propose the following release candidate (RC0) of Apache
99+
Arrow version #{@release_version}. This is a release consisting of #{n_resolved_issues}
100+
resolved GitHub issues[1].
101+
102+
This release candidate is based on commit:
103+
#{@current_commit} [2]
104+
105+
The source release rc0 is hosted at [3].
106+
The binary artifacts are hosted at [4][5][6][7][8][9].
107+
The changelog is located at [10].
108+
109+
Please download, verify checksums and signatures, run the unit tests,
110+
and vote on the release. See [11] for the SHA-512 checksum for this RC and [12]
111+
for how to validate a release candidate.
112+
113+
See also a verification result on GitHub pull request [13].
114+
115+
The vote will be open for at least 72 hours.
116+
117+
[ ] +1 Release this as Apache Arrow #{@release_version}
118+
[ ] +0
119+
[ ] -1 Do not release this as Apache Arrow #{@release_version} because...
120+
121+
[1]: https://github.com/apache/arrow/issues?q=is%3Aissue+milestone%3A#{@release_version}+is%3Aclosed
122+
[2]: https://github.com/apache/arrow/tree/#{@current_commit}
123+
[3]: https://dist.apache.org/repos/dist/dev/arrow/apache-arrow-#{@release_version}-rc0
124+
[4]: https://packages.apache.org/artifactory/arrow/almalinux-rc/
125+
[5]: https://packages.apache.org/artifactory/arrow/amazon-linux-rc/
126+
[6]: https://packages.apache.org/artifactory/arrow/centos-rc/
127+
[7]: https://packages.apache.org/artifactory/arrow/debian-rc/
128+
[8]: https://packages.apache.org/artifactory/arrow/ubuntu-rc/
129+
[9]: https://github.com/apache/arrow/releases/tag/apache-arrow-#{@release_version}-rc0
130+
[10]: https://github.com/apache/arrow/blob/#{@current_commit}/CHANGELOG.md
131+
[11]: #{tarball_hash}
132+
[12]: https://arrow.apache.org/docs/developers/release_verification.html
133+
[13]: #{verify_pr_url || "null"}
134+
TEMPLATE
135+
end
136+
end

0 commit comments

Comments
 (0)