Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Remove use of deprecated record_batches_to_json_rows function from arrow-json #24981

Open
hiltontj opened this issue May 9, 2024 · 3 comments
Assignees
Labels

Comments

@hiltontj
Copy link
Contributor

hiltontj commented May 9, 2024

We are using a function from arrow-json crate that is deprecated: record_batches_to_json_rows

It is used in the following places:

  1. &arrow_json::writer::record_batches_to_json_rows(batches.as_slice())?,
  2. let json_rows = record_batches_to_json_rows(&[&batch])

The deprecation warning indicates that we should be using Writer. Doing so in the latter case should be easier since we are serializing directly to the bytes outputted in the API response, however, the former may be a bit trickier, as we are modifying the serialized JSON values before serializing to bytes in the response.

@hiltontj hiltontj added the v3 label May 9, 2024
@JeanArhancet
Copy link

I am interested in taking on this task. Is it possible to assign this ticket to me?

@hiltontj
Copy link
Contributor Author

hiltontj commented Jun 6, 2024

Hi @JeanArhancet - that would be awesome, thank you! This issue is not in our immediate backlog, so if you would like to tackle it by opening a PR, you are welcome to do that.

My recommendation is to start by solving (1.) in the issue description, and opening a PR for only that. Since that is using arrow-json's writer for its intended purpose: serializing and writing JSON. Specifically, I would look at arrow-json's ArrayWriter, as that serializes to a JSON array, as opposed to JSON Lines. Separately, we do intend to support JSON Lines (see #24654).

As for (2.), the use of record_batches_to_json_rows is a bit of a hack there, so the solution is not to use the arrow-json writer, but will be something different.

Please keep in mind that you will need to read and sign our CLA in order for contributions to be accepted. You can find that here: https://www.influxdata.com/legal/cla/

@JeanArhancet
Copy link

Thanks for your feedback, @hiltontj.

I've opened a PR for the first part: #25046

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

No branches or pull requests

2 participants