Add horizontal scaling roadmap by mdpearson · Pull Request #5652 · openboxes/openboxes · GitHub
Skip to content

Add horizontal scaling roadmap#5652

Open
mdpearson wants to merge 1 commit into
developfrom
horizontal-scaling
Open

Add horizontal scaling roadmap#5652
mdpearson wants to merge 1 commit into
developfrom
horizontal-scaling

Conversation

@mdpearson

Copy link
Copy Markdown
Member

This PR is just a shell for a large markdown document detailing how I think we can support horizontally scaling the OpenBoxes Tomcat instance.

Covers 14 areas including Quartz JDBC migration, Redis sessions, cache consolidation, connection pool sizing, load balancing, etc.

Ideally after initial review I'd pull this to a Jira epic and make a story for each of the bullet points within it, then start whittling away at them.

Copilot AI review requested due to automatic review settings December 3, 2025 21:17
@github-actions github-actions Bot added the domain: documentation Changes or discussions relating to documentation label Dec 3, 2025
@mdpearson mdpearson self-assigned this Dec 3, 2025
Covers 14 areas including Quartz JDBC migration, Redis sessions,
cache consolidation, connection pool sizing, load balancing, etc.

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR adds comprehensive documentation for enabling horizontal scaling of the OpenBoxes Tomcat application. It introduces two detailed markdown documents outlining the technical challenges, solutions, and implementation steps needed to support running multiple application instances concurrently.

Key Changes:

  • Added high-level scaling roadmap covering 14 areas of concern (SCALING.MD)
  • Added detailed implementation plan with code examples and phase-by-phase guidance (HORIZONTAL_SCALING_MIGRATION_PLAN.md)
  • Documented critical blockers including Quartz JDBC migration, Redis sessions, analytics job coordination, and load balancing configuration

Reviewed changes

Copilot reviewed 1 out of 1 changed files in this pull request and generated no comments.

File Description
SCALING.MD High-level roadmap document identifying 14 areas requiring changes for horizontal scaling, organized by priority (critical blockers, important improvements, unresolved questions, and non-issues)
HORIZONTAL_SCALING_MIGRATION_PLAN.md Detailed implementation guide with 9 phases, including code samples, configuration examples, Ansible playbooks, acceptance criteria, and deployment checklists for horizontal scaling migration

Note: This review identified several critical issues that should be addressed before using this documentation for implementation, including inline AI assistant directives that should be removed, configuration inconsistencies, bugs in code examples, and security concerns around credential management.


💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

@codecov

codecov Bot commented Dec 3, 2025

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 9.69%. Comparing base (1bb7314) to head (126fa9c).
⚠️ Report is 114 commits behind head on develop.

Additional details and impacted files
@@             Coverage Diff              @@
##             develop   #5652      +/-   ##
============================================
+ Coverage       9.12%   9.69%   +0.56%     
- Complexity      1170    1319     +149     
============================================
  Files            701     717      +16     
  Lines          45281   45778     +497     
  Branches       10851   10926      +75     
============================================
+ Hits            4131    4436     +305     
- Misses         40497   40656     +159     
- Partials         653     686      +33     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

@jmiranda

jmiranda commented Dec 18, 2025

Copy link
Copy Markdown
Member

@ewaterman @awalkowiak

What are your thoughts on the best place for this document?

This artifact should be public-facing and is a living document (proposal, discussion)

The options are:

  • Project root (current PR) - doesn't seem to fit here since it's a living document at the moment
  • RoadMap (https://openboxes.com/roadmap/) - doesn't really fit here but wanted to provide the link
  • Official Documentation (docs.openboxes.com) - maybe some day
  • Contributor Guide (https://openboxes.gitbook.io/contributor-guide/) - probably not here
  • Community Forum (https://community.openboxes.com/) - i like this option because it hits a larger audience
  • GitHub Discussions - make sense here since the audience is more deverloper-oriented. discussion happens here until we want to take action and then perhaps we create an ADR based on decisions made
  • ADR Repository (docs/adr or a separate ADR repo) - eventually, the decisions we make should go here

I forgot two other options, but these might not be valid at the moment for the reasons expressed.

  • Confluence PIH - not public facing
  • Confluent OpenBoxes - no one really uses it at the moment

I think my recommendation (unless and until we create a project / tickets for this work), would be to move the discussion to Discourse (community.openboxes.com) so more eyes see it and provide feedback or GitHub Discussions since it's a developer proposal / discussion at the moment.

And then if we actually standup an ADR repo we can create separate ADRs for each decision we need to make so there's a record.

cc @mdpearson

@jmiranda

jmiranda commented Dec 18, 2025

Copy link
Copy Markdown
Member

Some more thoughts. I think there's a state machine here that would be cool to implement with respect to these types of proposals. This isn't just a random idea I had one day. It's been brewing for a long time, but I haven't had time or budget to work on it.

See the Opporuntity Backlog (challenges, opportunities) that I created a few years ago.
https://docs.google.com/spreadsheets/d/1DDt58XYPdmWbnWeYnrZTLntb4RVIRBAJd4VercP1OYI/edit?usp=sharing

The challenge is listed in the following document re: Challenges > HRDL-015 Scalability and there are a few opportunities mentioned in the Opportunities tab

  • REF-008 Quartz clustering configuration PoC
  • REF-009 Stateless API
  • REF-010 Session Replication
  • REF-100 Database per tenant multi-tenancy

I realized recently that we probably need to make some incremental progress on the horizontal scaling challenge because we have a few implementations that might eventually reach limits re: vertical scaling.

This Opportunity Backlog also probably needs to be public-facing (separate repo?) to track all of the challenges we've identified. These could live in GitHub issues but given how much work Evan has done cleaning that issue tracker, that's probably a bad idea. I think there could one place for identifying / tracking the challenges and opportunities (like the Google Sheet) and another when we promote that idea to be implemented.

So here's the potential flow for all of this:

  • Document the "challenge" that we face (see Challenges tab)
  • Create an Opporuntity Backlog item that might require discovery / discussion
  • Promote to a Roadmap Item when we think we might be able to start thinking about the opportunity
  • Create a Discussion in Github Discussions or Discourse (what we currently have here)
  • Create a Detailed Proposal to detail the scope of each issue (should try to limit this to each challenge)
  • Create any Decision Records (ADR) for each decision that we need to make
  • Add Product / Technical Roadmap Item (confirmed MVP scope) to show progress
  • Create a GitHub project / issues for the scope of work to be implemented
  • Update the Roadmap Item (include status, version, last updated, link to proposal or project depending on status)

@ewaterman

Copy link
Copy Markdown
Member

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

domain: documentation Changes or discussions relating to documentation

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants