[dpdk-ci] March 25, 2021 Community CI Meeting Minutes

[Lincoln Lavoie]

##########################################################
Attendees

1. Lincoln Lavoie
2. Brandon Lo
3. Aaron Conole
4. Ali Alnubani
5. Ashley Weltz
6. David Liu
7. Juraj Linkeš
8. Michael Santana
9. Lijuan
10. Thomas Monjalon

##########################################################
Agenda

1. Performance Testing Metrics
2. CI Status
3. Bugzilla Status
4. Test Development
5. Any other business

##########################################################
Minutes

==========================================================
Performance Testing Metrics

* UNH-IOL has tested the DTS patch on most of the bare-metal systems in the
lab.  Couple of systems require python updates.  Requests / feedback made
on output format, to align with data type expectations, etc.
* Lijuan is planning to send the V2 patch for next week.  Migration won’t
start until after the next DPDK CI meeting, to allow everyone time to
review and test the changes and latest DTS.
* Migration is estimated for about 2-3 days, including taking systems
offline, making updates, and putting back into production, to move all of
the systems to reporting performance numbers. Other testing should be able
to remain running, except for during the database migration processes (this
is for safety, so we have a stable backup copy if necessary).
* Database migration will be run to create percentage results where
possible.  Data prior to January 2020 was not storing the full absolute
results in the database (only the delta), so it’s not possible to convert
to the percentage, so that data will not show up in the Grafana charts,
etc.  If the group wants, we can maintain the earlier (current) carts for
that data with the delta values.  Performance numbers for LTS19.11 may be
worth recreating into the new system (percentage).  We will not maintain
the graphs / access to the older data.

==========================================================
CI Status

----------------------------------------------------------
UNH-IOL Community Lab

* Reviewed and tested DTS changes for migration to percentage, along with
development of the migration planning (i.e. review of script changes,
database migration, etc.).
* Continued migration process of compile testing to containers.
** New containers for Ubuntu 20.04 and CentOS Stream 8
** Existing virtual machines are now being migrated over to containers
** Unit testing is being investigated for implementation into the dashboard
* Checked SPDK failures, fixed through changes on DPDK main branch.
* Tracked down performance failures on arm to a change made while debugging
the intermittent timeout failures.  This has been rolled back, but we have
not been able to track down the other intermittent timeout failure.  Aaron
will help coordinate a session to debug with the arm team on this issue.
Mellanox requests to rerun the patches that are waiting in patchwork, due
to the previous failure on the performance run.
* Aaron / Lincoln to update bugzilla to track Q1 SOW items.
* How to request / schedule a rerun of a patch?  What is the process?
** Add patches to a list (i.e. google doc), remove them when the rerun is
either started or completed.  How could this be automated?  Button in
patchwork?
** In the meantime, email the lab or the ci list to make rerun requests.


----------------------------------------------------------
Intel Lab

* Submitting test results per patch for DPDK, failures were being missed by
patchwork, this has been fixed.
* Results being sent for DTS to patchworks, reports do show up within DTS
patchwork, but the reports show up in the same email list as DPDK reports
with the same tags, so it may be confusing to the developers.  Thomas / Ali
will look into solutions in April.

----------------------------------------------------------
Travis CI / Github Actions / OBS

* Aaron is still working on the OBS monitor script.  Before this can be
turned on, we need to clear the backlog from the OBS system (might be
easiest to delete / recreate the account), so the storage is cleared out.
* Where do we want to store the scripts for github / obs?  Currently
separate and used by multiple projects (i.e. open vSwtich, etc.).  Could
this be handled as documentation as part of the ci repo (i.e. a markdown
readme, etc).

==========================================================
Bugzilla Status

*
https://bugs.dpdk.org/buglist.cgi?bug_status=UNCONFIRMED&bug_status=CONFIRMED&bug_status=IN_PROGRESS&columnlist=product%2Ccomponent%2Cpriority%2Cbug_status%2Cassigned_to%2Cshort_desc%2Cchangeddate&component=job%20scripts&component=UNH%20infra&component=Intel%20Lab&component=Travis%20CI&list_id=3419&order=priority%2Cchangeddate%20DESC&product=lab&query_format=advanced&resolution=---

==========================================================
Test Development

* Request to share the SOW priorities for 2021, this could be handled as a
part of the Q1 blog as well.
* Q1 Priorities in the UNH-IOL Community Lab
   * Migration of compile / unit jobs into containers, increase unit
testing coverage / runs where possible, increase ABI coverage to all OS
types except Windows
   * Roll out the percentage performance changes
   * Migrate to templated pipelines (i.e. utilize patch categorization to
direct testing)
   * Investigate Coverity alternative (build list for tech board review)
   * Provide report on job run times to techboard (i.e. how long testing
jobs take)

==========================================================
Any other business

* April 8, 2021

-- 
*Lincoln Lavoie*
Principal Engineer, Broadband Technologies
21 Madbury Rd., Ste. 100, Durham, NH 03824
lylavoie at iol.unh.edu
https://www.iol.unh.edu
+1-603-674-2755 (m)
<https://www.iol.unh.edu>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mails.dpdk.org/archives/ci/attachments/20210325/2ddd52cd/attachment.htm>