Batch API progress data

[bwbroersma]

There is an undocumented progress query parameter in the List requests and Request status:

Internet.nl/interface/batch/util.py

Lines 871 to 885 in 6d587d8

	provide_progress = request.GET.get("progress")
	provide_progress = provide_progress and provide_progress.lower() == "true"
	batch_requests = BatchRequest.objects.filter(user=user).order_by("-id")[:limit]
	batch_info = []
	for batch_request in batch_requests:
	request_dict = batch_request.to_api_dict()
	if provide_progress:
	total_domains = BatchDomain.objects.filter(batch_request=batch_request).count()
	finished_domains = BatchDomain.objects.filter(
	batch_request=batch_request, status__in=(BatchDomainStatus.done, BatchDomainStatus.error)
	).count()
	request_dict["progress"] = f"{finished_domains}/{total_domains}"
	request_dict["num_domains"] = total_domains
	batch_info.append(request_dict)

Internet.nl/interface/batch/util.py

Lines 911 to 921 in 6d587d8

	def get_request(request, batch_request, user):
	provide_progress = request.GET.get("progress")
	provide_progress = provide_progress and provide_progress.lower() == "true"
	res = {"request": batch_request.to_api_dict()}
	if provide_progress:
	total_domains = BatchDomain.objects.filter(batch_request=batch_request).count()
	finished_domains = BatchDomain.objects.filter(
	batch_request=batch_request, status__in=(BatchDomainStatus.done, BatchDomainStatus.error)
	).count()
	res["request"]["progress"] = f"{finished_domains}/{total_domains}"
	res["request"]["num_domains"] = total_domains

Does the ORM of this result in two (heavy) SQL SELECT COUNT(*) queries? (heavy because count-* must check the visibility of each row)

Should this option be documented or enabled by default for non-finished domains? (and maybe the domain count can be added to the metadata?)

[bwbroersma]

Assigned to self, steps:

• Run batch in develop
• Check PostgreSQL queries (by enable logging)
• Fill relevant database tables with dummy data
• Explain queries to estimate impact