View of all repositories on Github and Gitlab that have Crystal code in them.

Installation

• Install Crystal
• Clone this repository
• Install dependencies shards install
• Rename .env.example to .env, and set variables.
• Run Web server as crystal src/web.cr

Development

psql -c 'CREATE DATABASE shards_info_development;' -U postgres
crystal src/cli.cr migrate

Repository counts

Repository listings read dependency, dependent, and fork counts from the repository_statistics materialized view. A Mosquito job refreshes it every 10 minutes; the worker must be running for counts to stay current. Newly added repositories show zero counts until the next successful refresh. Repository details and visibility still come from the live tables.

Run migrations before starting the updated web and worker binaries. The Lustra migration creates and populates the view together with the unique index needed for concurrent refreshes. The view is managed by migrations, rather than Lustra::View.register, so subsequent migration runs preserve its index.

To refresh counts manually:

REFRESH MATERIALIZED VIEW CONCURRENTLY public.repository_statistics;

Database Operations (Makefile)

We use a Makefile to automate database schema dumps and restoration. Available commands:

Dump database schema and migrations

make db-dump

This dumps both the schema and migration metadata (__lustra_metadatas table) into src/db/structure.sql.

Dump schema only

make db-dump-schema

Dump migrations metadata only

make db-dump-migrations

Restore from dump

make db-restore

Using a different database

make db-dump DB_NAME=your_database_name
make db-restore DB_NAME=your_database_name

View all available commands

make help

Frontend

Install depencencies:

npm install

After modifications run npm run build, npm run oxlint, and npm run oxfmt.

Specs

Specs create and migrate their own PostgreSQL test database via spec/initdb.cr. The default connection settings are:

DATABASE_URL=postgres://postgres:postgres@localhost/shards_info_test
POSTGRES_HOST=localhost
POSTGRES_USER=postgres
POSTGRES_PASSWORD=postgres
POSTGRES_DB=postgres

Redis must also be running locally for cache/session/Mosquito/Defense dependencies. The app uses these defaults in test:

REDIS_URL=redis://localhost:6379/0
SESSION_REDIS_URL=redis://localhost:6379/1
MOSQUITO_REDIS_URL=redis://localhost:6379/2
DEFENSE_REDIS_URL=redis://localhost:6379/3

Run specs with:

KEMAL_ENV=test crystal spec

To only recreate the test database:

KEMAL_ENV=test crystal spec/initdb.cr

Special thanks

• Crystal language
• Lustra - Advanced ORM between PostgreSQL and Crystal
• Kemal - Web microframework for Crystal
• Mosquito - A generic background task runner for Crystal applications
• raven.cr - Crystal client for Sentry
• cr-cmark-gfm - Crystal C bindings for cmark-gfm
• noir - Syntax Highlight Library for Crystal
• Logo icon taken from Game Icons pack under CC BY 3.0 license.

Contributing

1. Fork it (https://github.com/mamantoha/shards-info/fork)
2. Create your feature branch (git checkout -b my-new-feature)
3. Commit your changes (git commit -am 'Add some feature')
4. Push to the branch (git push origin my-new-feature)
5. Create a new Pull Request

Contributors

• mamantoha Anton Maminov - creator, maintainer