Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Improve JSONPath eval performance #3018

Open
jcamiel opened this issue Jul 5, 2024 · 0 comments
Open

Improve JSONPath eval performance #3018

jcamiel opened this issue Jul 5, 2024 · 0 comments
Labels
enhancement New feature or request

Comments

@jcamiel
Copy link
Collaborator

jcamiel commented Jul 5, 2024

In packages/hurl/src/jsonpath/eval/query.rs:

impl Query {
    /// Eval a JSONPath `Query` for a `serde_json::Value` input.
    /// It returns an Option<`JsonResultPath`>.
    pub fn eval(&self, value: &serde_json::Value) -> Option<JsonpathResult> {
        let mut result = JsonpathResult::SingleEntry(value.clone());
        for selector in &self.selectors {
            match result.clone() {
                JsonpathResult::SingleEntry(value) => {
                    result = selector.eval(&value)?;
                }
                JsonpathResult::Collection(values) => {
                    let mut elements = vec![];
                    for value in values {
                        match selector.eval(&value)? {
                            JsonpathResult::SingleEntry(new_value) => {
                                elements.push(new_value);
                            }
                            JsonpathResult::Collection(mut new_values) => {
                                elements.append(&mut new_values);
                            }
                        }
                        result = JsonpathResult::Collection(elements.clone());
                    }
                }
            }
        }
        Some(result)
    }
}

The first thing we do is cloning the input value:

    pub fn eval(&self, value: &serde_json::Value) -> Option<JsonpathResult> {
        let mut result = JsonpathResult::SingleEntry(value.clone());

For every jsonpath query, the input value is the root document. For a big JSON response, it means cloning the response for every query, even if we need a small node. As we've serde_json::Value as input and a result containing serde_json::Value, maybe we can work exclusivley with references on the input.

Check https://github.com/Orange-OpenSource/hurl/blob/master/integration/hurl/tests_ok/parse_cache.hurl, we're evaluating ~80 JSONPath query on a 1.3M JSON document in 180 ms (at 380 ms):

Screenshot 2024-07-05 at 14 00 31

@jcamiel jcamiel added the enhancement New feature or request label Jul 5, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request
Projects
None yet
Development

No branches or pull requests

1 participant