[R] Redesigned xgboost() interface skeleton

[david-cortes]

ref #9810

This PR starts off the redesigned interface for xgboost(), by adding a bare-bones version that replaces the old one:

• Uses x/y interface instead of xgb.DMatrix.
• Enforces a match between the type and shape of y and the objective.
• Makes the objective optional, picking a default according to the type of y.
• Allows column name matching in parameters that refer to columns.
• Uses xgb.QuantileDMatrix whenever possible.
• Keeps extra metadata about y (column names and factor levels).
• Inherits from xgb.Booster, but has an additional class xgboost, so that the current methods can be used on it.

But:

• Doesn't yet make use of the additional metadata that it keeps in other methods, beyond the new print that's added here.
• Doesn't yet have a more idiomatic predict method.
• Has not yet updated examples and guides to use this function.
• Doesn't yet deal with the unidiomatic passing of params.
• Doesn't yet offer some functionalities that it should, like custom metrics.
• Has not yet updated the serialization doc for this new class.

Among others. It should nevertheless provide a base to work on.

A couple items I am not so sure about - would be helpful to get reviews on these:

• Does the function in this PR correctly capture all the possible cases in which there can be more than one score per leaf? (e.g. multinomial, multi-task).
• Is it treating Surv objects correctly for all cases?
• Where can I find reference papers to add to the docs explaining how XGBoost implements things like monotonicity constraints and interaction contrstraints? I see there's the original paper that introduces XGBoost that is mentioned in the docs, but I imagined that a lot more might have been written by the same authors over time.

CCing @mayer79 for review.

[trivialfis]

Thank you for sharing the early work. I will look into the new example and introduction for learning about the new interface.

[david-cortes]

Thank you for sharing the early work. I will look into the new example and introduction for learning about the new interface.

To clarify: there's no intro and only a very small example, since there's no specialized predict yet for this class, for example.

[trivialfis]

Would you like me to add a feature branch in xgboost for you to iterate on?

[david-cortes]

Would you like me to add a feature branch in xgboost for you to iterate on?

Thanks but I don't think there's any need for that. Or are you planning a 2.1.1 release soon?

[mayer79]

Awesome work @david-cortes !

The auto-selection of the objective is really neat.

While reading the code, I added some detail stuff (not that they are important, it was just to keep me busy).

Some additional comments:

• I'd prefer helper functions like process.x.and.col.args() to be named like _process_x_and_col_args().
• We will also need to deal with X (store names of features, store factor levels of character/factor columns).

[mayer79]

Suggested change

	if (is.array(base_margin) && length(dim(base_margin)) > 2) {
	if (is.array(base_margin) && length(dim(base_margin)) > 2L) {

[trivialfis]

Any issue with these parameters?

[david-cortes]

They are meant to be passed as arguments to the function xgboost() instead of being passed through params.

[trivialfis]

monotone_constraints and interaction_constraints might be better as a part of normal params? This way it feels more consistent. It's difficult from a user's perspective to understand why some of these parameters are different.

[david-cortes]

From my perspective:

• Some parameters I chose to put as function arguments in order to give them a default value visible to the user (e.g. nthreads) or make the default value change according to the data (e.g. objective).
• Some parameters are independent of the data (e.g. regularization) or work regardless of whether the data supports them or not (e.g. max categ. to one-hot), while others refer specifically to columns in the data (e.g. interaction constraints) and would not work with a different dataset.
• Ideally, later on there should be a function like xgb.control or similar that would list the parameters and offer auto-completion by having them as function arguments (e.g. like VGAM::vglm.control or glmx::glmx.control). Those train-control parameters typically involve things that are independent of the data.
• Alternatively, and as is more common in R, all parameters could be moved to function arguments (e.g. like ranger::ranger or glmnet::glmnet), but perhaps that would make the docs very hard to use.

It's difficult from a user's perspective to understand why some of these parameters are different.

Yes, it's a bit arbitrary, but that is also the situation in the other interfaces - e.g. the number of boosting rounds is a function argument, as is verbose_eval in the python interface. It also feels like parameters such as seed should be in the train control list, but that would have other side effects due to the logic introduced in an earlier PR (#10029).

[david-cortes]

Perhaps for v3 you could consider having a division between data-dependent parameters (e.g. feature_weights is passed under DMatrix, while monotone_contraints is passed under params) and data-independent parameters.

[trivialfis]

Thank you for the detailed explanation. Please think of me as a newcomer instead of a maintainer. Or think of the future GitHub issues we will need to reply to when someone puts the parameter in the wrong place and asks why.

but that is also the situation in the other interfaces

I think that's only true for the low-level interface, and some of them are only in this way because we think of it as an unnecessary breaking change. For higher-level interfaces like the scikit-learn, or Spark, they have a relatively consistent interface. For the scikit-learn, we are moving the parameters (slowly, can find some examples like early_stoppping_rounds and metric) according to the scikit-learn estimator guideline where the constructor takes hyper-parameter and sometimes feature-related data like feature types, while the fit method takes data(sample)-dependent parameters that can be split for early stopping, etc.

default value visible to the user

We can compromise on this, maybe the document is sufficient?

Alternatively, and as is more common in R, all parameters could be moved to function arguments

That sounds reasonable to me. Personally, I prefer consistency, which has been a pain point for XGBoost for a while now. Would be great to see improvement in this area.

Perhaps for v3 you could consider having a division between data-dependent parameters (e

Hmm, since this is a new interface already, maybe it's better to do it right the first time if we know where the issue is? The internal implementation can be changed anytime we need, but changing the interface is particularly difficult, especially with the CRAN policy.

[david-cortes]

Ok then, I'll move all parameters to function arguments, leaving them as undocumented ... for now (to be changed in a later PR). We'll see how the docs and examples end up looking later on.

[david-cortes]

Modified it by removing params and making all parameters as function arguments, by using ... for now.

[trivialfis]

Could you please add some comments for these indexing?

[david-cortes]

Added a comment block above.

[trivialfis]

Is this going to make single thread the default for R xgboost?

[david-cortes]

No, the default is to use all threads, as obtained when calling parallel::detectCores().

[trivialfis]

XGBoost takes 0 as using all available cores.

[david-cortes]

Changed the condition and added it to the docs.

[trivialfis]

We can support COO internally, for CSC, maybe we can simply transpose in R. Do you think it can help the make code less complex?

[david-cortes]

Maybe, but what would be the benefit in such case if the data still needs to be duplicated?

[david-cortes]

Actually I just remembered that constructing the simple DMatrix copies the data, so yes there would be one less copy in such case.

[david-cortes]

Modified it to accept COO and CSC for QDM, with a casting for now until direct QDM construction from COO is added.

[trivialfis]

Do we need to worry about data.table and potentially other libraries that inherit from these known types in this error message?

[david-cortes]

No, because all data.table objects are also data.frames.

class(data.table())

[1] "data.table" "data.frame"

[trivialfis]

I'm aware of this, hence the question, is it possible that this error message might lead the user to think data.frame is not supported? Or it's common knowledge in the R world?

[david-cortes]

No, because if class(x) contains data.frame as an entry, inherits will return TRUE, so there will be no error message in a derived class like data.table or tibble. And if it is a class like arrow or polars DataFrame which do not inherit from data.frame, then the error will show as expected, since they are not supported.

[trivialfis]

Thank you for the excellent work, will merge once the CI passes.