Add fail-fast flag to src batch

[gabtorre]

This adds a new --fail-fast flag to the src batch commands. When enabled, any errors encountered while executing steps in a repository will stop the execution of the whole batch spec. The flag is off by default, so existing behavior is preserved.

• A new CancelableContext type was added to support cancellation with errors. The batch commands now wrap the context with this to enable fail-fast behavior.
• Updated contextCancelOnInterrupt to support cancellation with error causes.
• Added error check in executeBatchSpec to surface fail-fast cancellation errors.
• Errors now propagate up by calling the cancel context's Cause()
• Minor code reorganizations around log file handling to return logs if -fail-fast and -keep-logs are used together.

Test plan

Updated tests.

[BolajiOlajide]

Hey @gabtorre

Apologies for the long delay. I'll review this soon.
Thank you so much.

[BolajiOlajide]

Left over debug statement

[BolajiOlajide]

This is awesome. TIL I learned about the .WithCancelCause method.

[BolajiOlajide]

I wonder if we should be passing in nil directly here, an error could occur at any point in the execution of a batch spec.

Is there a reason why the handler function doesn't return a named return value and we can then pass that into defer cancel?

handler := func(args []string) (err error) {
	...
	ctx, cancel := contextCancelOnInterrupt(context.Background())
	defer cancel(err)
	...
}

This way we are guaranteed to always have access to whatever error occurs in the lifecycle of an execution.

[BolajiOlajide]

Is there a need for this struct? Is there a reason we can't pass in the context and failFastCancel as separate arguments?

Or better still have the failFastCancel as one of the fields in executeBatchSpecOpts, that way we are certain ctx is always going to be of type context.Context and wouldn't be confusing to anyone in the future;

[BolajiOlajide]

Since the arguments to this method is getting bulky, we can move it into a struct and pass in the cancel function explicitly.

Suggested change

	func (c *Coordinator) ExecuteAndBuildSpecs(ctx CancelableContext, batchSpec *batcheslib.BatchSpec, tasks []*Task, ui TaskExecutionUI) ([]*batcheslib.ChangesetSpec, []string, error) {
	type ExecuteAndBuildSpecsArgs struct {
	batchSpec *batcheslib.BatchSpec
	tasks []*Task
	ui TaskExecutionUI
	cancelFunc context.CancelCauseFunc
	}
	func (c *Coordinator) ExecuteAndBuildSpecs(ctx context.Context, args ExecuteAndBuildSpecsArgs) ([]*batcheslib.ChangesetSpec, []string, error) {

[BolajiOlajide]

The cancelFunc isn't being used here, I wonder if we need to pass it in here as an argument?