Skip to content

refactoring views out of upgrade db schema scripts - #9401

Open
KlausDornsbach wants to merge 4 commits into
apache:mainfrom
scclouds:refactoring-db-upgrade-views
Open

KlausDornsbach wants to merge 4 commits into
apache:mainfrom
scclouds:refactoring-db-upgrade-views

Conversation

@KlausDornsbach

@KlausDornsbach KlausDornsbach commented Jul 16, 2024 •

Copy link
Copy Markdown
Contributor

Description

Based on the comment #9385 (comment) from @DaanHoogland, and considering that all the DB view definitions are currently refactored out of the DB upgrade schema sql scripts, I took the liberty to delete most of the remaining unused view definitions on these scripts, as they are overwritten in the end of the DB upgrade workflow.

Types of changes

  • Breaking change (fix or feature that would cause existing functionality to change)
  • New feature (non-breaking change which adds functionality)
  • Bug fix (non-breaking change which fixes an issue)
  • Enhancement (improves an existing feature and functionality)
  • Cleanup (Code refactoring and cleanup, that may add test cases)
  • build/CI

Feature/Enhancement Scale

  • Major
  • Minor

How Has This Been Tested?

Tested resetting the database to the 4.0.0 version via the management script cloudstack-setup-databases. After the database reset, the management server was also reset to ensure it ran all upgrade scripts.

Once the reset process was completed, various API operations were tested, particularly those that use database views to build their responses. Verifying the behavior was the same after the changes. During these tests, normal behavior was consistently observed. All API operations performed as expected, with no errors encountered.

Furthermore, the log stream was monitored throughout the testing phase to detect any potential issues or anomalies. The logs remained clear, showing no errors or warnings related to the current refactor.

@KlausDornsbach KlausDornsbach mentioned this pull request Jul 16, 2024
14 tasks
@DaanHoogland

Copy link
Copy Markdown
Contributor

@blueorangutan package

@blueorangutan

Copy link
Copy Markdown

@DaanHoogland a [SL] Jenkins job has been kicked to build packages. It will be bundled with KVM, XenServer and VMware SystemVM templates. I'll keep you posted as I make progress.

@codecov

codecov Bot commented Jul 16, 2024 •

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 20.69%. Comparing base (b215abc) to head (e2f2250).
⚠️ Report is 2148 commits behind head on main.

Additional details and impacted files
@@              Coverage Diff              @@
##               main    #9401       +/-   ##
=============================================
+ Coverage     15.55%   20.69%    +5.14%     
- Complexity    12010    21773     +9763     
=============================================
  Files          5500     6492      +992     
  Lines        481850   590805   +108955     
  Branches      61843    72461    +10618     
=============================================
+ Hits          74935   122274    +47339     
- Misses       398628   455252    +56624     
- Partials       8287    13279     +4992     
Flag Coverage Δ
uitests 3.88% <ø> (-0.29%) ⬇️
unittests 22.00% <ø> (+5.67%) ⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

@DaanHoogland DaanHoogland left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

clgtm, @KlausDornsbach I trust you checked these views actually exist? I know I did a round of checking when I extended the views directory a while ago, but we can never be too sure ;)

@blueorangutan

Copy link
Copy Markdown

Packaging result [SF]: ✔️ el7 ✔️ el8 ✔️ el9 ✔️ debian ✔️ suse15. SL-JID 10381

@DaanHoogland

Copy link
Copy Markdown
Contributor

@blueorangutan test

@blueorangutan

Copy link
Copy Markdown

@DaanHoogland a [SL] Trillian-Jenkins test job (centos7 mgmt + kvm-centos7) has been kicked to run smoke tests

@blueorangutan

Copy link
Copy Markdown

[SF] Trillian Build Failed (tid-10883)

@KlausDornsbach

Copy link
Copy Markdown
Contributor Author

Thanks for the heads up, did a double check and things are looking fine

@blueorangutan

Copy link
Copy Markdown

[SF] Trillian test result (tid-10905)
Environment: kvm-alma8 (x2), Advanced Networking with Mgmt server a8
Total time taken: 49093 seconds
Marvin logs: https://github.com/blueorangutan/acs-prs/releases/download/trillian/pr9401-t10905-kvm-alma8.zip
Smoke tests completed. 135 look OK, 2 have errors, 0 did not run
Only failed and skipped tests results shown below:

Test Result Time (s) Test File
test_02_add_multiple_annotations Failure 2.26 test_annotations.py
test_06_purge_expunged_vm_background_task Failure 341.12 test_purge_expunged_vms.py

@yadvr

yadvr commented Jul 19, 2024

Copy link
Copy Markdown
Member

LGTM, but I'm not sure if it would break anything that may require the views to be created during db upgrade paths are executed and before the views are created (or re-created). I also don't know if this needs upgrade tests across different versions & hypervisor combinations. Worth investigating them, before merging this.

@DaanHoogland

Copy link
Copy Markdown
Contributor

LGTM, but I'm not sure if it would break anything that may require the views to be created during db upgrade paths are executed and before the views are created (or re-created). I also don't know if this needs upgrade tests across different versions & hypervisor combinations. Worth investigating them, before merging this.

good point about the upgrades, @rohityadavcloud . I'll start at least from 4.19 and 4.18. We can always advice to upgrade stepwise via any of those first.

@DaanHoogland

Copy link
Copy Markdown
Contributor

@rohityadavcloud , both upgradespassed in the an automated upgrade test with test VMs etc. Is this good enough you think?

@github-actions

Copy link
Copy Markdown

This pull request has merge conflicts. Dear author, please fix the conflicts and sync your branch with the base branch.

@DaanHoogland

Copy link
Copy Markdown
Contributor

@KlausDornsbach , can you resolve the conflicts, please?

@DaanHoogland

Copy link
Copy Markdown
Contributor

@blueorangutan package

@blueorangutan

Copy link
Copy Markdown

@DaanHoogland a [SL] Jenkins job has been kicked to build packages. It will be bundled with KVM, XenServer and VMware SystemVM templates. I'll keep you posted as I make progress.

@blueorangutan

Copy link
Copy Markdown

Packaging result [SF]: ✔️ el8 ✔️ el9 ✖️ debian ✔️ suse15. SL-JID 10758

@DaanHoogland

Copy link
Copy Markdown
Contributor

I though we already had this behind us, is it still needed @KlausDornsbach? cc @vladimirpetrov

@DaanHoogland

Copy link
Copy Markdown
Contributor

@blueorangutan package

@blueorangutan

Copy link
Copy Markdown

@DaanHoogland a [SL] Jenkins job has been kicked to build packages. It will be bundled with KVM, XenServer and VMware SystemVM templates. I'll keep you posted as I make progress.

@weizhouapache

Copy link
Copy Markdown
Member

@blueorangutan package

@blueorangutan

Copy link
Copy Markdown

@weizhouapache a [SL] Jenkins job has been kicked to build packages. It will be bundled with no SystemVM templates. I'll keep you posted as I make progress.

@blueorangutan

Copy link
Copy Markdown

Packaging result [SF]: ✔️ el8 ✔️ el9 ✔️ el10 ✔️ debian ✔️ suse15. SL-JID 17761

@DaanHoogland

Copy link
Copy Markdown
Contributor

@blueorangutan test

@blueorangutan

Copy link
Copy Markdown

@DaanHoogland a [SL] Trillian-Jenkins test job (ol8 mgmt + kvm-ol8) has been kicked to run smoke tests

@blueorangutan

Copy link
Copy Markdown

[SF] Trillian Build Failed (tid-16091)

@blueorangutan

Copy link
Copy Markdown

Packaging result [SF]: ✔️ el8 ✔️ el9 ✔️ el10 ✔️ debian ✔️ suse15. SL-JID 17923

@DaanHoogland

Copy link
Copy Markdown
Contributor

@blueorangutan test

@blueorangutan

Copy link
Copy Markdown

@DaanHoogland a [SL] Trillian-Jenkins test job (ol8 mgmt + kvm-ol8) has been kicked to run smoke tests

@blueorangutan

Copy link
Copy Markdown

[SF] Trillian test result (tid-16143)
Environment: kvm-ol8 (x2), zone: Advanced Networking with Mgmt server ol8
Total time taken: 51075 seconds
Marvin logs: https://github.com/blueorangutan/acs-prs/releases/download/trillian/pr9401-t16143-kvm-ol8.zip
Smoke tests completed. 150 look OK, 1 have errors, 0 did not run
Only failed and skipped tests results shown below:

Test Result Time (s) Test File
test_02_add_multiple_annotations Failure 2.23 test_annotations.py

@nvazquez

Copy link
Copy Markdown
Contributor

Hi @KlausDornsbach @DaanHoogland @sureshanaparti what is the status of this PR? Is it ready for testing?

@DaanHoogland

Copy link
Copy Markdown
Contributor

@nvazquez , imnsho it should have been merged already.

@DaanHoogland DaanHoogland moved this from Backlog to Ready in CloudStack Testing Aug 31, 2026
@sureshanaparti sureshanaparti moved this from Ready to In review in CloudStack Testing Sep 7, 2026
@sureshanaparti sureshanaparti moved this from In review to In progress in CloudStack Testing Sep 7, 2026
@sureshanaparti

Copy link
Copy Markdown
Contributor

Hi @KlausDornsbach can you check and address the outstanding comments. thanks.

@DaanHoogland

Copy link
Copy Markdown
Contributor

@sureshanaparti @KlausDornsbach has not been very active lately, to my knowledge. Not sure if he is still involved in the project. I think you'll have to do the changes yourself???

@nvazquez

Copy link
Copy Markdown
Contributor

Hi @JoaoJandre @winterhazel @hsato03 not sure if Klaus is still active, can you please advise if you plan to have any further additions to this PR or we can apply the review suggestion and proceed the review?

@JoaoJandre

Copy link
Copy Markdown
Contributor

Hi @JoaoJandre @winterhazel @hsato03 not sure if Klaus is still active, can you please advise if you plan to have any further additions to this PR or we can apply the review suggestion and proceed the review?

Hello @nvazquez Klaus is no longer active. I'll apply the suggestions and push a new commit

@JoaoJandre

Copy link
Copy Markdown
Contributor

@sureshanaparti @nvazquez can you review?

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

Status: In progress

Development

Successfully merging this pull request may close these issues.

9 participants