Bug 40526

Summary:	wrong output of gb-task-gen-changelog (as if old changelog was empty)
Product:	Infrastructure	Reporter:	Ivan Zakharyaschev <imz>
Component:	girar	Assignee:	placeholder <placeholder>
Status:	NEW ---	QA Contact:	Andrey Cherepanov <cas>
Severity:	normal
Priority:	P5	CC:	glebfm, ldv
Version:	unspecified
Hardware:	x86_64
OS:	Linux
URL:	https://lists.altlinux.org/pipermail/sisyphus-incominger/2021-July/622505.html

Description Ivan Zakharyaschev 2021-07-17 23:48:56 MSK

https://lists.altlinux.org/pipermail/sisyphus-incominger/2021-July/622505.html
https://lists.altlinux.org/pipermail/sisyphus-incominger/2021-July/622524.html

The whole history is printed, as if the old changelog was empty (or nothing matched):

2021-Jul-17 17:40:26 :: plan: src +1 -1 =17536, noarch +1 -1 =16223
#100 emacs-text-modes 0.2-alt3 -> 0.2-alt4
 Sat Jul 17 2021 Ivan Zakharyaschev <imz at altlinux> 0.2-alt4
 - For markdown-mode:
   + Match also *.md, *.mdwn, and *.markdown files in auto-mode-alist
     (additionally to *.text files) (ALT#30484).
     (These extensions are used on GitHub, in ikiwiki, and GitLab.)
   + Match these patterns only at the end of filenames.
 Sat Dec 26 2009 Terechkov Evgenii <evg at altlinux.ru> 0.2-alt3
 - Markdown-mode added (ALT #20383)
 - Spec cleanup
 Sat Feb 04 2006 Eugene Vlasov <eugvv at altlinux.ru> 0.2-alt2
 [...]
2021-Jul-17 17:40:26 :: emacs-text-modes: closes bugs: 30484 20383

Running rpm_changes_since function from gb-sh-functions locally on the same pair of packages can't reproduce this problem.

The line "#100 emacs-text-modes 0.2-alt3 -> 0.2-alt4" indicates that rpm_changes_since was called ($F0 is not empty), given the following code of gb-task-gen-changelog script:

join -t$'\t' -o 1.1,1.2,1.3,1.4,1.5,2.3 plan/{add,rm}-src > $tmpdir/src
sort -t$'\t' -u -k1,3 -o $tmpdir/src{,} plan/add-src

while F0=; read -r N EVR F P I F0; do
	changelog="build/$I/changelog"
	if [ -n "$F0" ]; then
		# Must be executed before commit-repo!
		srpm0="$GB_REPO_DIR/files/SRPMS/$F0"
		rpm_changes_since "$P" "$srpm0"
	else
		rpm_recent_changes "$P"
	fi < /dev/null > $tmpdir/log
	grep '^[[:space:]]*[^[:space:]]' < $tmpdir/log |
		sed "s/@$qdom_in>/@$qdom_out>/g;s/^/ /" > "$changelog"
	if [ -n "$F0" ] && [ -s "$changelog" ]; then
		# Report it
		evr0="${F0%.src.rpm}"
		evr0="${evr0#$N-}"
		printf '#%s %s %s -> %s\n' \
			"$I" "$N" "$evr0" "$EVR"
		head < "$changelog"
		if [ "$(wc -l < "$changelog")" -gt 10 ]; then
			echo ' [...]'
		fi
	fi
done < $tmpdir/src

That's very puzzling: as if rpm -qp on the old package printed something strange and different from my local tests, or gawk behaved differently.

Comment 1 Ivan Zakharyaschev 2021-07-17 23:52:33 MSK

To test rpm_changes_since function from gb-sh-functions locally, one can simply clear the gb-sh-conf script:

> gb-sh-conf

and set tmpdir.

Like this:

/bin/sh -f -c 'export PATH="$PATH:$PWD"; . gb-sh-functions ||:; set -eu; tmpdir=/home/imz/tests/test-girar-rpm_changes_since; rpm_changes_since /tasks/279434/build/100/x86_64/srpm/emacs-text-modes-0.2-alt4.src.rpm /ALT/Sisyphus/files/SRPMS/emacs-text-modes-0.2-alt3.src.rpm'; echo $?

Comment 2 Ivan Zakharyaschev 2021-07-19 16:45:01 MSK

It could, for example, be something like hdrcache[1] that affected the result of rpm -qp in girar and made it different from my local tests. However, this kind of special system configuration (hdrcache or something else affecting rpm -qp) can not be seen from girar.spec, so we can only guess.

[1]: hdrcache is present in Sisyphus, p9, p8:

$ fgrep hdrcache  /ALT/{Sisyphus,p{9,8}}/{noarch,x86_64}/base/contents_index
/ALT/Sisyphus/x86_64/base/contents_index:/usr/lib64/rpm-plugins/hdrcache.so	rpm-plugin-hdrcache
/ALT/Sisyphus/x86_64/base/contents_index:/usr/lib64/rpmhdrcache.so	rpmhdrcache
/ALT/p9/x86_64/base/contents_index:/usr/lib64/rpm-plugins/hdrcache.so	rpm-plugin-hdrcache
/ALT/p9/x86_64/base/contents_index:/usr/lib64/rpmhdrcache.so	rpmhdrcache
/ALT/p8/x86_64/base/contents_index:/usr/lib64/rpmhdrcache.so	rpmhdrcache
$

Comment 3 Ivan Zakharyaschev 2021-07-19 19:38:12 MSK

(Ответ для Ivan Zakharyaschev на комментарий #1)
> To test rpm_changes_since function from gb-sh-functions locally, one can
> simply clear the gb-sh-conf script:
> 
> > gb-sh-conf
> 
> and set tmpdir.
> 
> Like this:
> 
> /bin/sh -f -c 'export PATH="$PATH:$PWD"; . gb-sh-functions ||:; set -eu;
> tmpdir=/home/imz/tests/test-girar-rpm_changes_since; rpm_changes_since
> /tasks/279434/build/100/x86_64/srpm/emacs-text-modes-0.2-alt4.src.rpm
> /ALT/Sisyphus/files/SRPMS/emacs-text-modes-0.2-alt3.src.rpm'; echo $?

It's simpler to call rpm_changes_since() and do this test with the commit 66c2d2e in master branch in my Git repo, where I moved the "pure" functions (in the same sense as the gb-x-* helpers) into a new file, gb-x-sh-functions.

/bin/sh -efu -c 'export PATH="$PATH:$PWD"; . gb-sh-functions;
tmpdir=/home/imz/tests/test-girar-rpm_changes_since; rpm_changes_since
/tasks/279434/build/100/x86_64/srpm/emacs-text-modes-0.2-alt4.src.rpm
/ALT/Sisyphus/files/SRPMS/emacs-text-modes-0.2-alt3.src.rpm'; echo $?

(No need to clean gb-sh-conf, because it is not loaded and needed by any gb-x-*.)

Comment 4 Dmitry V. Levin 2021-07-30 13:17:45 MSK

(In reply to Ivan Zakharyaschev from comment #2)
> It could, for example, be something like hdrcache[1] that affected the
> result of rpm -qp in girar and made it different from my local tests.
> However, this kind of special system configuration (hdrcache or something
> else affecting rpm -qp) can not be seen from girar.spec, so we can only
> guess.
> 
> [1]: hdrcache is present in Sisyphus, p9, p8:
> 
> $ fgrep hdrcache  /ALT/{Sisyphus,p{9,8}}/{noarch,x86_64}/base/contents_index
> /ALT/Sisyphus/x86_64/base/contents_index:/usr/lib64/rpm-plugins/hdrcache.so
> rpm-plugin-hdrcache
> /ALT/Sisyphus/x86_64/base/contents_index:/usr/lib64/rpmhdrcache.so
> rpmhdrcache
> /ALT/p9/x86_64/base/contents_index:/usr/lib64/rpm-plugins/hdrcache.so
> rpm-plugin-hdrcache
> /ALT/p9/x86_64/base/contents_index:/usr/lib64/rpmhdrcache.so	rpmhdrcache
> /ALT/p8/x86_64/base/contents_index:/usr/lib64/rpmhdrcache.so	rpmhdrcache
> $

No,
$ rpmquery rpmhdrcache rpm-plugin-hdrcache
package rpmhdrcache is not installed
package rpm-plugin-hdrcache is not installed

Comment 5 Ivan Zakharyaschev 2021-07-30 17:07:37 MSK

(Ответ для Dmitry V. Levin на комментарий #4)

> No,
> $ rpmquery rpmhdrcache rpm-plugin-hdrcache
> package rpmhdrcache is not installed
> package rpm-plugin-hdrcache is not installed

Thanks for information!

Could you try to run a command like the one below (no special privileges must be needed) and see whether this bad behavior can be reproduced?

(Ответ для Ivan Zakharyaschev на комментарий #3)

> /bin/sh -efu -c 'export PATH="$PATH:$PWD"; . gb-sh-functions;
> tmpdir=/home/imz/tests/test-girar-rpm_changes_since; rpm_changes_since
> /tasks/279434/build/100/x86_64/srpm/emacs-text-modes-0.2-alt4.src.rpm
> /ALT/Sisyphus/files/SRPMS/emacs-text-modes-0.2-alt3.src.rpm'; echo $?

It's very puzzling: what can the reason for the difference in behavior be?

Comment 6 Ivan Zakharyaschev 2021-07-30 17:10:12 MSK

Minor correction (regarding "gb-x-sh-functions"):

/bin/sh -efu -c 'export PATH="$PATH:$PWD"; . gb-x-sh-functions;
tmpdir=/home/imz/tests/test-girar-rpm_changes_since; rpm_changes_since
/tasks/279434/build/100/x86_64/srpm/emacs-text-modes-0.2-alt4.src.rpm
/ALT/Sisyphus/files/SRPMS/emacs-text-modes-0.2-alt3.src.rpm'; echo $?