Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: better user experience around ibis examples #9110

Open
1 task done
chloeh13q opened this issue May 2, 2024 · 4 comments
Open
1 task done

feat: better user experience around ibis examples #9110

chloeh13q opened this issue May 2, 2024 · 4 comments
Labels
examples Issues/PRs related to `ibis.examples` feature Features or general enhancements mssql The Microsoft SQL Server backend mysql The MySQL backend postgres The PostgreSQL backend

Comments

@chloeh13q
Copy link
Contributor

chloeh13q commented May 2, 2024

Is your feature request related to a problem?

I've been playing around with ibis examples a lot lately, and I noticed that some examples are broken on different backends, and some backends don't support examples at all. It was hard to get an idea of what works and what doesn't, other than trying it out manually and see whether I get an error or not.

What is the motivation behind your request?

Create a smoother user experience around ibis examples, so that the examples can be utilized better for experimentation and exploration around ibis.

Describe the solution you'd like

  • Fix the backends where example loading is broken (in progress; so far we have identified issues with postgres, mysql, mssql).
  • Have an actual error message if the current default backend doesn't support examples (Druid, PySpark, RW). That way we can error cleanly, and avoid downloading the example parquet file that can't be loaded.

What version of ibis are you running?

main

What backend(s) are you using, if any?

No response

Code of Conduct

  • I agree to follow this project's Code of Conduct
@chloeh13q chloeh13q added the feature Features or general enhancements label May 2, 2024
@lostmygithubaccount
Copy link
Member

this might be too janky, but once #8115 is in, perhaps for those that don't support it if you have DuckDB installed we could use that to load them in, then transfer them over?

@gforsyth
Copy link
Member

gforsyth commented May 2, 2024

At least for postgres, mysql, and mssql, the issues loading examples are symptomatic of general problems with NaN/NULL handling and temp table creation, which we should fix anyway.

xref #8792 #9095

@gforsyth gforsyth added postgres The PostgreSQL backend mysql The MySQL backend mssql The Microsoft SQL Server backend examples Issues/PRs related to `ibis.examples` labels May 2, 2024
@chloeh13q
Copy link
Contributor Author

chloeh13q commented May 2, 2024

Okay, I wrote up a script to do some more systematic testing, and I think there are some other edge cases that we're not properly handling on different backends. Going to keep a list of examples that are broken:

MySQL:

  • PitchingPost
  • PitchingPost_raw
  • nycflights13_flights
  • nycflights13_weather
  • wowah_data_raw

Dask:
None

DataFusion:

  • nycflights13_flights
  • nycflights13_weather
  • wowah_data_raw

Exception: DataFusion error: NotImplemented("Unsupported SQL type Timestamp(Some(6), None)") on all three

sqlite:

  • lakers

ProgrammingError: Error binding parameter 4: type 'datetime.time' is not supported

But also I wonder if this is something we'd want to test in CI.

@chloeh13q
Copy link
Contributor Author

chloeh13q commented May 3, 2024

Can confirm that all the examples works in postgres with the NaN -> None fix.

ClickHouse examples are also broken with the error

IbisInputError: Cannot specify both `temp=True` and `overwrite=True` for ClickHouse

which seems like it could be a simple fix.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
examples Issues/PRs related to `ibis.examples` feature Features or general enhancements mssql The Microsoft SQL Server backend mysql The MySQL backend postgres The PostgreSQL backend
Projects
Status: backlog
Development

No branches or pull requests

3 participants