Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Scheduler to fetch data from Git repositories #645

Open
sduenas opened this issue Apr 1, 2024 · 0 comments
Open

Scheduler to fetch data from Git repositories #645

sduenas opened this issue Apr 1, 2024 · 0 comments

Comments

@sduenas
Copy link
Member

sduenas commented Apr 1, 2024

The current version of the platform needs of several instances for analyzing more than 5000 data sources. For example, for a project with around 3500 high-activity repositories that retrieves data from GitHub (commits, issues and pull requests), the platform needs 3 days to start analyzing new data.

The goal is to start working on a new scheduler that allows to scale the platform according to what's defined on the current version of the roadmap.

The first iteration of this scheduler will be to add support for fetching data from Git repositories.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
Status: Near Term
Development

No branches or pull requests

1 participant