summaryrefslogtreecommitdiffstats
path: root/utils
AgeCommit message (Collapse)AuthorFilesLines
2019-06-19utils/job-prereq.py: Use yaml safe_loadhalibut_7.99.3halibut_7.99.2halibut/7.99.3halibut/7.99.27.99.37.99.2Corentin LABBE1-1/+1
On my gentoo, using yaml.load now give: Traceback (most recent call last): File "./utils/job-prereq.py", line 81, in <module> main() File "./utils/job-prereq.py", line 66, in main job_yaml = yaml.load(job) File "/usr/lib64/python2.7/site-packages/yaml/__init__.py", line 109, in load raise RuntimeError("Unsafe load() call disabled by Gentoo. See bug #659348") RuntimeError: Unsafe load() call disabled by Gentoo. See bug #659348 Note that on recent ubuntu, a warning appears also. (like on AGL jenkins) /w/workspace/ci-platform-meta-agl-demo-verify-CIBT-qemuarm64/repoclone/releng-scripts/utils/job-prereq.py:66: YAMLLoadWarning: calling yaml.load() without Loader=... is deprecated, as the default Loader is unsafe. Please read https://msg.pyyaml.org/load for full details. job_yaml = yaml.load(job) This is due to a security risk of using yaml.load() Since releng-script does not rely on any behavour provided by load(), let's convert the call to safe_load(). This will also clean/reduce the output of jenkins logs Change-Id: I6158dcc21c0f7ec4645fb7b16acfaf1a6963cb26 Signed-off-by: Corentin LABBE <clabbe@baylibre.com>
2019-06-19Fix create-jobs for snapshot/daily/weekly build-typeKhouloud Touil1-0/+4
This patch take care of the snapshot/daily/weekly build-types if they are requested without mentioning the branch and version args. Bug-AGL: SPEC-2471 Change-Id: I86e5465162cc20be82b33feedcbed7f78300053d Signed-off-by: Khouloud Touil <ktouil@baylibre.com>
2019-06-06agljobtemplate: fix url generation for m3ulcbCorentin LABBE1-2/+4
When adding support for upsquare the URL generation for m3ulcb was broken. This patchs fix the generation by adding a new variable for the build name. Fixes: 32b88dfd726b ("SPEC-2075: fix URL generation for upsquare non-CI build") Bug-AGL: SPEC-2470 Signed-off-by: Corentin LABBE <clabbe@baylibre.com> Change-Id: I84bc2d8bbcbc49b048bbd95f84a531bde9ac6aba
2019-06-06Add support for qemuarm/qemuarm64 in utils/job-prereq.pyCorentin LABBE1-1/+1
This patchs permits utils/job-prereq.py to work with qemuarm/qemuarm64 Bug-AGL: SPEC-2376 Signed-off-by: Corentin LABBE <clabbe@baylibre.com> Change-Id: Ie8615a9ab4d1f0053afc098db508814e6aad1bf7
2019-05-30add tags: section, ensure tags are always lowercaseKevin Hilman1-1/+0
Whenever --device-tags are passed, a "tags:" section is added to the job with all of the tags passed on the command-line. To facilitate this, the device_tags list is passed directly from python to the jinja templates, and iteration is done in jinja instead of a string created in python. Also, while here, ensure that tags are always passed lower-cased. IMPORTANT: this implies that any tags in LAVA labs that are upper-case will *never* be used. Change-Id: I8f274098a69e34c32c5ed2d1c430eca0d364bcc5 Signed-off-by: Kevin Hilman <khilman@baylibre.com>
2019-05-22SPEC-2075: fix URL generation for upsquare non-CI buildCorentin LABBE1-1/+1
When adding support for upsquare, I have added a new machine_frag_url which contains the machine to use in url. It works for CI build type, but for other I miss to convert one machine to machine_frag_url Bug-AGL: SPEC-2075 Change-Id: Ieaaf71b62425cf73fb9582b84666640c4e9d6cae Signed-off-by: Corentin LABBE <clabbe@baylibre.com>
2019-05-15SPEC-1850: permit to override master branch for appfwCorentin LABBE2-2/+5
I have assumed that only master is used for appfw but it is false. Some app review are done against non-master branch. This patch adds a way to change the branch used via --app-branch. Bug-AGL: SPEC-1850 Change-Id: I10c8ad0e5a08158f9c2295692e4b3c266b97b14e Signed-off-by: Corentin LABBE <clabbe@baylibre.com>
2019-05-03utils/agljobtemplate.py: Fix APPURL generationCorentin LABBE1-8/+9
The generated APPURL was invalid: - it needed to be always using ci - it missed branch and arches path This patch move the generation of the application url in the jinja file for accessing the new sdk_arch Bug-AGL: SPEC-1850 Change-Id: Ie3ffff5f6181fdc89c3ebbd41078e4d60a06992b Signed-off-by: Corentin LABBE <clabbe@baylibre.com>
2019-05-03Remove porterCorentin LABBE1-1/+1
As asked by JSM on review 21232, this patch removes the porter machine. Bug-AGL: SPEC-2375 Change-Id: Iaa8083a271bb61ee1f9208f671d617fc15a00028 Signed-off-by: Corentin LABBE <clabbe@baylibre.com>
2019-04-18Permit to generate jobs for the upsquareCorentin LABBE1-3/+5
This patch adds suppport for the upsquare. Upsquare use a GRUB boot method, so we needed to add a new generic-grub-tftp boot template. Since all x86_64 use the same yocto generated build name, we add a new upsquare machine, but tweaked a bit utils/agljobtemplate.py to keep intel-corei7-64 everywhere (except for the LAVA device-type). Bug-AGL: SPEC-2075 Change-Id: I15c850fada53efbcda4fd58527d7b7414865f89d Signed-off-by: Corentin LABBE <clabbe@baylibre.com>
2019-04-16Modify the releng-scripts to deal with branches and dtb namesKhouloud Touil2-5/+14
This patch takes the branch and machine arguments and decide which is the dtb name to be used. Change-Id: I09bd67de51920c81780a77be396906a15d8247f5 Signed-off-by: Khouloud Touil <ktouil@baylibre.com>
2019-04-11Implement SPEC-1850: Add application-lifecycleCorentin LABBE2-2/+25
This patch adds a new parameter application_url which permit to adds a location where to download apps and test them. The applications_url could also be generated via app-changeid/app-patchset. Change-Id: I88d91b5e6e30b5ab5d72e8181b15b0f561bea72d Signed-off-by: Corentin LABBE <clabbe@baylibre.com>
2019-04-11Fix the use of the --build-type argumentKhouloud Touil1-1/+1
This patch fixes the use of the --build-type to take only the first part of it, which is the build type. Change-Id: Ic2f631a639babbd075bb005b2d33415e4ca916ab Signed-off-by: Khouloud Touil <ktouil@baylibre.com>
2019-03-22Fix the job parametersKhouloud Touil1-2/+8
This patch fixes the job parameters, if the build type is ci the parameters will be parsed as changeid and patchset otherwise it will be parsed as branch and version. Change-Id: I3f9abfd91a1b551974dd82ea82dc97a2e0a11d95 Signed-off-by: Khouloud Touil <ktouil@baylibre.com>
2019-03-15create-jobs: fix the error checkKhouloud Touil1-2/+2
This patch fixes the error check to permit the job creation when inserting only the --url arguement and not the branch/version or the changeid/patchsetm, to grant the job creation with a custom url. Change-Id: I625e13cacee8da6df569ab90038318c0994ec1c6 Signed-off-by: Khouloud Touil <ktouil@baylibre.com>
2019-03-07job-prereq: fix use of render_job after updatesKevin Hilman1-2/+2
Change-Id: I094220e829686f9acd362339fea93f947f5bfa57 Signed-off-by: Kevin Hilman <khilman@baylibre.com>
2019-02-28Modify the url_fragmentKhouloud Touil1-1/+3
Following all the modification and to keep things correct, this patch modify the url_fragment to gave a correct url following the build type if it is 'ci' or 'release' or other. Change-Id: I7122d5c4c31e37337ca68af3d86313d36e5b8f5b Signed-off-by: Khouloud Touil <ktouil@baylibre.com>
2019-02-28Add new metadata git.urlKhouloud Touil1-0/+6
This patch adds a new metadata field git.url, this is a link back to the AGL gerrit. Change-Id: I7f7b5fff79198fac7814004ab99b3687cf92125c Signed-off-by: Khouloud Touil <ktouil@baylibre.com>
2019-02-28Add new --build-id argumentKhouloud Touil2-2/+7
This patch adds a new build-id arguement that will fill the 'kernel_defconfig' in the metadata field with the build id extracted from the build-info file. Change-Id: I77bcc253a41d67d97afd7db32b8c8560d4309145 Signed-off-by: Khouloud Touil <ktouil@baylibre.com>
2019-02-28Reuse the build_version to fill the kernel_versionKhouloud Touil1-17/+4
The build_version will fill the kernel_version with the right value, this value will be the 'DISTRO_VERSION' extracted from the build-info file. Change-Id: Iecfdfd587dc15d53ea872d45b1916d33f6e382bf Signed-off-by: Khouloud Touil <ktouil@baylibre.com>
2019-02-28Use the --branch argument differentlyKhouloud Touil2-11/+5
Change the use of the --branch argument to display in the vsc_branch metadata field the AGL branch used in the build and get ride of the hard coded value "agl-branch". Change-Id: I9ff371248569c40ab9980347da14110e5227e1db Signed-off-by: Khouloud Touil <ktouil@baylibre.com>
2019-02-28Add new vcs_commit argumentKhouloud Touil2-2/+7
Add new vcs_commit arguement that will display in the metadeta field the build hash information. Change-Id: I079dfb8efda2ce82677e867804b3937d5a1f13c2 Signed-off-by: Khouloud Touil <ktouil@baylibre.com>
2019-02-28Separate --branch/--changeid --version/--patchsetKhouloud Touil2-11/+19
This patch separates the --branch/--changeid and --version/--patchset arguments to use each one of them differently. As well it renames url_branch to changeid and url_version patchset. It modifies also the error check to follow the other changes and be sure that the needed arguments are well inserted. Change-Id: If64ae3620b1519fc391e79d954247938cffcb491 Signed-off-by: Khouloud Touil <ktouil@baylibre.com>
2019-01-24Add new device-tags featureKhouloud Touil2-3/+7
utils/create-jobs: Add the --device-tags argument Add new --device-tags arguement to create a job that contain the device tags as parameters. utils/agljobtemplate: Add the device_tags to the job Add the device_tags to the job in order to be used later in the test scripts to determine which test case should be runned or not. Change-Id: If22fb40b2ddf1a7c0496a59d4318e633028495b9 Signed-off-by: Khouloud Touil <ktouil@baylibre.com>
2018-12-05templates: callbacks: support multiple callback targetsLoys Ollivier2-5/+7
LAVA jobs can now submit multiple callbacks per job. Update the create-jobs tool and its underlying templates to support this feature. - Callback templates are now split in two file and follow LAVA specification: 'section' and 'subsection'. - create-jobs tool supports multiple arguments for the '--callback-to' option. - Default configuration for callback targets can now be a list of targets. Change-Id: I36c502c0bb5b54010ce32a53ba78a6be5e6a1642 Signed-off-by: Loys Ollivier <lollivier@baylibre.com>
2018-11-21templates: add job generation support for iPXE based boardsguppy_6.99.1guppy/6.99.16.99.1Liu Wenlong1-6/+14
Since iPXE based boards can use NBD boot[1], now in releng-scripts, we also add support to generate LAVA jobs for iPXE based boards. With this support, Intel-UP2, one of the community reference boards, can do validation tests with the generated jobs easily. [1] https://git.lavasoftware.org/lava/lava/merge_requests/64/diffs Change-Id: I9adce67d7e9207181675e82bf22a653c73682eb8 Signed-off-by: Liu Wenlong <liuwl.fnst@cn.fujitsu.com>
2018-10-15agljobtemplate: fix UnboundLocalError for build_verson_defaultKevin Hilman1-0/+1
Change-Id: If8c422f8da34d6ab3474200095aff1f8cb7923d0 Signed-off-by: Kevin Hilman <khilman@baylibre.com>
2018-10-06agljobtemplate: CI builds: rename branch to gerritKevin Hilman1-1/+1
Change-Id: I8341bb7c3ab65a02a7b144171f778c850484850b Signed-off-by: Kevin Hilman <khilman@baylibre.com>
2018-10-06agljobtemplate: allow cmdline build-version to overrideKevin Hilman1-4/+5
Allow the --build-version cmdline option to override the other ways of generating the kernel.version metadata. Change-Id: I9da1de7fdc795d888a7b864965755821a53e2830 Signed-off-by: Kevin Hilman <khilman@baylibre.com>
2018-09-28Extend timeouts, add callback and gitignoreJan-Simon Möller1-1/+1
Change-Id: Ib28d25e1efd0033458fa9ae787610a8de9f5d275 Signed-off-by: Jan-Simon Möller <jsmoeller@linuxfoundation.org>
2018-09-27Add shortcut script for ci testsJan-Simon Möller1-0/+19
call with ./run-flounder-ci-tests 5.99.x lava-tool needs the auth-token setup. Signed-off-by: Jan-Simon Möller <jsmoeller@linuxfoundation.org>
2018-09-12fix metadata for ci build-typeguppy_6.90.0guppy/6.90.06.90.0Kevin Hilman1-2/+14
When using --build-type=ci, the url_branch and url_version are repurposed for gerrit changeid and patchset. However, this means that the kernelCI metadata fields "branch" and "version" values are interpreted as integers by the backend, causing various problems. To add clarity to kernelCI fields, prefix with "changeid-" and "patchset-", and this also makes kernelci-backend interpret fields as strings, as expected. Change-Id: I6b648d9c94751a3ce20ec48c593d60b7e71640b1 Signed-off-by: Kevin Hilman <khilman@baylibre.com>
2018-06-26utils/agljobtemplate: fixup image_type, branch and versionKevin Hilman1-0/+7
Fixup LAVA job metadata that is used by kernelCI UI for displaying branch, and version info, and also set "image-type" to the AGL build type (e.g., gerrit, snapshot, release) Change-Id: I12ca93dfbe8629bde5dc35ab1f056f8331805e40 Signed-off-by: Kevin Hilman <khilman@baylibre.com>
2018-01-18job-prereq: Fix the tool after build changesLoys Ollivier2-2/+2
The job-prereq tool got broken by some latest changes on the build/url scheme. Make the url argument for agljobtemplate optionnal. Pass build_type argument as build_type now that it is supported. Fixes: 788b31a122c3 ("utils: change the url / build behavior scheme") Change-Id: Ie9a0fc728a5e31cabd32e6463f2c6f913e1bd2cf Signed-off-by: Loys Ollivier <lollivier@baylibre.com>
2018-01-16Merge changes I44ae2990,Ied4e9f80,I9ce801a7Jan-Simon Moeller2-34/+43
* changes: templates: fix the rootfs name for m3ulcb and rpi3 daily builds utils: define test-plans based on build-type utils: change the url / build behavior scheme
2018-01-10templates: fix the rootfs name for m3ulcb and rpi3 daily buildsLoys Ollivier1-0/+1
The rootfs filename has changed for the daily builds. Change the default to the new filename for both platforms. Change-Id: I44ae29902d882ab6f38dc14bba11cc8aee88f8c2 Signed-off-by: Loys Ollivier <lollivier@baylibre.com>
2018-01-10utils: define test-plans based on build-typeLoys Ollivier2-19/+26
As the list of test is expending, we want to be able to specify test plans grouping several test definitions. These test plans are build type specific. Create a config file that specifies the test plans for each build type. If the user does not specify the tests to run then use the default ones for the build-type. Change-Id: Ied4e9f80d3e42787174189cd08499a2906e500ef Signed-off-by: Loys Ollivier <lollivier@baylibre.com>
2018-01-10utils: change the url / build behavior schemeLoys Ollivier2-23/+24
This is the first patch of the serie to differentiate builds and urls. As of now we were using the --url option to specify a build-type. Add a new option --build-type which specifies which type of build it is. From this build-type if no url is specified, the url can be defaulted by a configuration file. This configuration file is specific per user, e.g. AGL, others... Change-Id: I9ce801a7518b78ee859c6c3bbcad3a89e884e832 Signed-off-by: Loys Ollivier <lollivier@baylibre.com>
2018-01-03utils: Verify that callback options are correct v2eel_5.0.0eel/5.0.05.0.0Loys Ollivier1-2/+2
'--callback-from' is mandatory but not '--callback-to'. Fix this and allow specifying only 'from' without 'to'. Fixes: 5c6cf2bf9c09 ("utils: Verify that callback options are correct") Change-Id: I0706950cee4ede00cdbe7a3d1ade5500cc568502 Signed-off-by: Loys Ollivier <lollivier@baylibre.com>
2017-12-27utils: Miscellaneous pep8 changesLoys Ollivier2-20/+24
Coding style changes to follow pep8 coding guidelines. Change-Id: I9a9e542fb1e24da699639e0c3fd8bf98ad51aa79 Signed-off-by: Loys Ollivier <lollivier@baylibre.com>
2017-12-27utils: Verify that callback options are correctLoys Ollivier2-8/+11
When using callbacks, the user must specify '--callback-to' and '--callback-from'. Make these two arguments mutually dependant. If one is specified, the other becomes mandatory as well. Edit exceptions returns so they give meaningful information to the user in case something went wrong while reading from the callback configuration file. Change-Id: Iaa0de0b6f01ba605407db2f37c3da97aa0cacbc2 Signed-off-by: Loys Ollivier <lollivier@baylibre.com>
2017-12-07Merge "utils/agljobtemplate.py: Raise an error if jinja version is incorrect"eel_5.1.0eel_5.0.3eel_4.99.5eel_4.99.4eel/5.1.0eel/5.0.3eel/4.99.5eel/4.99.45.1.05.0.34.99.54.99.4eelKevin Hilman1-0/+4
2017-12-07utils/job-prereq.py: New tool to get LAVA job files nameLoys Ollivier1-0/+65
Add a new tool that outputs to the stdout the filenames needed by LAVA to run a test job. Required arguments are the machine type and the build type (e.g. CI, snapshots, release). Change-Id: I760fe59c6e04cf1e396c3772426fc8f6042f322c Signed-off-by: Loys Ollivier <lollivier@baylibre.com>
2017-12-06utils/agljobtemplate.py: Raise an error if jinja version is incorrectLoys Ollivier1-0/+4
The jinja2 template calls needs a jinja version >= 2.9. Otherwise fails to create a correct output yaml file. Check for the jinja version at runtime. More precisely, check for a specific function that was added in 2.9 and raise an error if it does not exist. Change-Id: Ifd627a25dcfabdd639c53e1b29b891db8bff50c4 Signed-off-by: Loys Ollivier <lollivier@baylibre.com>
2017-11-08Callback procedure updateLoys Ollivier2-15/+25
New callback procedure. Now the arguments are: - '--callback-from' LAVA lab - '--callback-to' kCI instance This new procedure helps differentiating the lab and the kCI target. Callback config files updated accordingly. Documentation on callback updated. Change-Id: I41b9dc86e33783f1d6a8164fc783557c81678c7f Signed-off-by: Loys Ollivier <lollivier@baylibre.com>
2017-11-07Fix AGL build versionLoys Ollivier1-3/+4
Fix the defaulted build version if not set from command line. Change-Id: Ia61dca57abad29cd9e1305bfead15e75c81e06f4 Signed-off-by: Loys Ollivier <lollivier@baylibre.com>
2017-11-06New machine, url, branch, version argumentslollivier2-20/+60
- Machine is now a mandatory argument: --machine - Change in the URL scheme usage: Now the argument is --url, it should be used with --branch & --version (or --changeid and --patchset which are aliases). The default url is the latest AGL release (dab - 4.0.2) - Build version is derived from the URL, branch and version if not set through command line. Example usage: ./utils/create-jobs.py --machine m3ulcb ./utils/create-jobs.py --machine qemux86-64 ./utils/create-jobs.py --url release --branch eel --version 4.99.1 --machine m3ulcb ./utils/create-jobs.py --url release --branch eel --version 4.99.1 --machine qemux86-64 ./utils/create-jobs.py --url daily --branch master --version latest --machine m3ulcb ./utils/create-jobs.py --url daily --branch master --version latest --machine raspberrypi3 ./utils/create-jobs.py --url ci --changeid 11533 --patchset 2 --machine raspberrypi3 ./utils/create-jobs.py --url ci --changeid 11533 --patchset 2 --machine m3ulcb ./utils/create-jobs.py --url http://baylibre.com/pub/agl/ci/raspberrypi3 --machine raspberrypi3 Update the README.md following changes on the url scheme and --machine argument. Bug-AGL: SPEC-975 Change-Id: Idd0d539627d56e8d28e1e4990570a929b3fa0f5d Signed-off-by: lollivier <lollivier@baylibre.com>
2017-10-06New arguments to specify build artifacts namesLoys Ollivier2-12/+24
Added new command line arguments to specifiy build artifacts names: - "--rootfs-img": for the rootfs file name - "--kernel-img": for the kernel name - "--dtb-img": for the dtb file name - "--modules-img": for the modules file name Change-Id: I665dee2b326aeaaf35148345f0501e49950ba9aa Signed-off-by: Loys Ollivier <lollivier@baylibre.com>
2017-10-04Add build_version command line optionLoys Ollivier2-2/+7
Add an option to specify the build version from the command line. The kernel_version in the job metadata is updated accordingly if this option is used. Otherwise the default kernel_version is used. Change-Id: Ifc73405d23cb48f48be3d558588bbd20b5063552 Signed-off-by: Loys Ollivier <lollivier@baylibre.com>
2017-10-03Revert "Add version command line option"Kevin Hilman2-7/+2
This reverts commit 2bc4acf27f841ee4e9554320ad03fa5d53f4b34b. Fails with: argparse.ArgumentError: argument --version/-v: conflicting option string(s): --version Change-Id: I93fa8b29f53f10e75c73833072d0d79700d31f22 Signed-off-by: Kevin Hilman <khilman@baylibre.com>