testing sometimes will fail or timeout · Issue #572 · timescale/tobs · GitHub
Skip to content
This repository was archived by the owner on Mar 20, 2026. It is now read-only.
This repository was archived by the owner on Mar 20, 2026. It is now read-only.

testing sometimes will fail or timeout #572

Closed

Description

@nhudson

What happened?

Running e2e tests sometimes hang/timeout or fail

How to reproduce it (as minimally and precisely as possible):

Run make start-kind && make cert-manager && make timescaledb-single && make e2e locally

  • Kubernetes cluster kind:

local cluster using kind make start-kind

Activity

  1. self-assigned this
    on Sep 1, 2022
  2. nhudson commented on Sep 1, 2022

    @nhudson
    ContributorAuthor

    Disabling all debug options with helm and chart-testing still can end up with a test that times out or fails.

    The timeout can occur when printing the log output while running make e2e. It always seems to eithe r get stuck/timeout after printing logs for timescaledb.

  3. nhudson commented on Sep 13, 2022

    @nhudson
    ContributorAuthor

    This is the issue I am seeing locally and with Github Actions reported here and then fixed here. I will attempt to remedy it by applying the new kubectl-timeout configuration option and see if that helps.

  4. nhudson commented on Sep 13, 2022

    @nhudson
    ContributorAuthor

    ct does seem to have big issues when printing out logs over a specific line count. Adding in the kubectl-timeout doesn't really help the issue. The main issue I can find with all various amount of testing is that ct will 100% stop working when printing out the logs from the timescaledb pod. This is the only pod which the logs are quite extensive. Even setting the logging level to 0, doesn't keep the logs from being printed so either way ct will just stop working indefinitely or until there is some timeout limit reached (320m) in Github Actions.

  5. nhudson commented on Sep 14, 2022

    @nhudson
    ContributorAuthor

    I created an issue upstream helm/chart-testing#471 that details what I am seeing with our use of ct.

    I also created PR to add the config option to disable printing of the logs to stdout here: helm/chart-testing#472

  6. nhudson commented on Sep 14, 2022

    @nhudson
    ContributorAuthor

    I have started using my patched version of ct with success with this tobs PR: #579

  7. nhudson commented on Dec 7, 2022

    @nhudson
    ContributorAuthor

    This has been fixed.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

Labels

bugSomething isn't working

Type

No type

Projects

No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions