WebQuery is a Pythonic way to interact with API's using SQL. In other words, Python + SQLite ≈ YQL.
- Dependencies:
- yaml
- lxml
- apsw
Interactive shell:
./webquery.py *.yml
Using a pipe:
echo "SELECT * FROM delicious_feeds_popular WHERE query='test' ORDER BY url DESC; SELECT url,title FROM microsoft_bing_web WHERE query='test' LIMIT 5;" | ./webquery.py *.yml
Limiting requests (useful for sorting):
echo "SELECT webquery('microsoft_bing_web','truncate',50); SELECT title,url FROM microsoft_bing_web WHERE query='sushi' ORDER BY title;" | ./webquery.py *.yml
In Python:
import apsw, webquery
db = apsw.Connection(':memory:')
webquery.attach(db)
cur = db.cursor()
cur.execute('CREATE VIRTUAL TABLE microsoft_bing_web USING webquery(microsoft.bing.web.yml);')
cur.execute('SELECT webquery(?,?,?);',('microsoft_bing_web','truncate',50))
for title,url in cur.execute('SELECT title,url FROM microsoft_bing_web WHERE query=? ORDER BY title;',('sushi',)):
print title,'>',url
In SQL:
-- load definition from a file
CREATE VIRTUAL TABLE table1 USING webquery(microsoft.bing.web.yml);
-- load definition from the Web
CREATE VIRTUAL TABLE table2 USING webquery(http://www.example.com/microsoft.bing.web.yml);
-- get configuration variable
SELECT webquery('table1','truncate');
-- set configuration variable and return previous value
SELECT webquery('table1','truncate',50);
- nb:
- The ORDER BY clause does not map to server-side sorting; it sorts the results that are returned by the server.
- Request truncation (by the truncate variable) always takes precedence over the LIMIT clause, preventing unbounded requests while performing a sorted query. It does not guarantee that the result set be smaller than the set value, so the LIMIT clause is still useful.