4

In the Linux terminal, I want to run a SPARQL query, and get as output the result as CSV:

$ query --service https://query.wikidata.org/sparql?query=SPARQL --sparql "SELECT ?i ?c WHERE { ?i wdt:P31 wd:Q3917681. ?i wdt:P137 ?c.}"
wd:Q4374016,wd:Q819
wd:Q4374028,wd:Q711
wd:Q4374063,wd:Q874
wd:Q4374039,wd:Q159

... or something similar.

Requirements:

  • Free, open source
  • Works with Wikidata
  • Works on Linux
  • Bonus if it can handle usual namespace prefixes without having to specify them on command line.
  • Having to write the request in a text file is acceptable too
2

I wrote this bash script to allow easy Wikidata queries from command line.

Usage:

./query-wikidata.sh "SELECT DISTINCT ?item WHERE {?item wdt:P31 wd:Q3917681. ?item wdt:P137 wd:Q16. ?item wdt:P131*/wdt:P17 wd:Q142.}"

OR this more readable syntax:

echo "
 SELECT DISTINCT
   ?item
 WHERE {
   ?item wdt:P31 wd:Q3917681.
   ?item wdt:P137 wd:Q16.
   ?item wdt:P131*/wdt:P17 wd:Q142.
 }
 " |./query-wikidata.sh

Output:

<?xml version='1.0' encoding='UTF-8'?>
<sparql xmlns='http://www.w3.org/2005/sparql-results#'>
        <head>
                <variable name='item'/>
        </head>
        <results>
                <result>
                        <binding name='item'>
                                <uri>http://www.wikidata.org/entity/Q137798</uri>
                        </binding>
                </result>
        </results>
</sparql>

Plus some network information on error output.

It knows the standard Wikidata prefixes.
License: GNU-GPLv3

2

For most applications the basic command-line approach is enough. Given the query in a file query.sparql, get CSV with curl:

curl -X POST https://query.wikidata.org/sparql -H "Accept: text/csv" --data-urlencode query@query.sparql

Here is another bash script that also supports JSON output, for instance to process with jq:

#!/bin/bash
#
# Run a Wikidata query from command line
# 
# Usage examples:
#
#   wdquery "DESCRIBE wd:Q42"
#   wdquery - xml < query.sparql
#   wdquery query.sparql json
#

QUERY=${1--}        # SPARQL query or file
FORMAT=${2-json}    # output format

ENDPOINT=https://query.wikidata.org/sparql

# make sure query is written to a file
if ! [ -f "$QUERY" ]
then
    TMPFILE="$(mktemp)"
    if [ "$QUERY" == "-" ]
    then
        # read query from STDIN
        cat > "$TMPFILE"
    else
        # write query to file
        echo "$QUERY" > "$TMPFILE"
    fi
    QUERY="$TMPFILE"
fi

curl -s -X GET -F "query=@$QUERY" "$ENDPOINT?format=$FORMAT"

[ -z "$TMPFILE" ] || rm "$TMPFILE"
1

It doesn't recognize Wikidata prefixes by default, and can't output CSV at the moment, but the RDF4J Console is a useful generic commandline application, not just for talking to Wikidata, but to any public SPARQL endpoint, your own RDF4J Server, or even a simple embedded triplestore of your own.

To install, you need a Java (8) Runtime Environment. Then, just download the RDF4J SDK. The console.sh script can be found in the bin/ directory.

To query Wikidata, start the console and add a new SPARQL endpoint definition:

./console.sh 
Connected to default data directory
RDF4J Console 2.0.1
Type 'help' for help.
> create sparql
Please specify values for the following variables:
SPARQL query endpoint: https://query.wikidata.org/sparql
SPARQL update endpoint: 
Local repository ID [endpoint@localhost]: wikidata
Repository title [SPARQL endpoint repository @localhost]: wikidata sparql endpoint
Repository created

Once that's set up, you can open the created repository (which is just a proxy for the public SPARQL endpoint):

> open wikidata
Opened repository 'wikidata'

Then just query (taking care not to overload the endpoint - use limit clauses if you're experimenting):

wikidata> select * where {?x rdf:type ?o } limit 10
Evaluating SPARQL query...
+-------------------------------------+-------------------------------------+
| x                                   | o                                   |
+-------------------------------------+-------------------------------------+
| <http://wikiba.se/ontology#Dump>    | <http://schema.org/Dataset>         |
| <http://www.wikidata.org/value/8000b1b66dfb1c74420cf59f0388d425>| <http://wikiba.se/ontology#GeoAutoPrecision>|
| <http://www.wikidata.org/value/8000d3ac475ff41c5e2e38ffda574d1c>| <http://wikiba.se/ontology#GeoAutoPrecision>|
| <http://www.wikidata.org/value/8000f71177f50273969eca2096803e29>| <http://wikiba.se/ontology#GeoAutoPrecision>|
| <http://www.wikidata.org/value/80018edb4bf3ecf344a2d4343d76235c>| <http://wikiba.se/ontology#GeoAutoPrecision>|
| <http://www.wikidata.org/value/800269618da5a8571140652500ade834>| <http://wikiba.se/ontology#GeoAutoPrecision>|
| <http://www.wikidata.org/value/8003743c7dbef48511f9df4e629fd462>| <http://wikiba.se/ontology#GeoAutoPrecision>|
| <http://www.wikidata.org/value/8003a8ceaf4f75884d2b9ff99e3c959c>| <http://wikiba.se/ontology#GeoAutoPrecision>|
| <http://www.wikidata.org/value/80062f03a3a91733e0eb81f50ca56208>| <http://wikiba.se/ontology#GeoAutoPrecision>|
| <http://www.wikidata.org/value/800682cb1e9157517ed01b1a4b1760bc>| <http://wikiba.se/ontology#GeoAutoPrecision>|
+-------------------------------------+-------------------------------------+
10 result(s) (1231 ms)

The above is all done interactively, but of course you can script these commands and then just feed them into the console.sh shellscript.

FWIW, output format specification is on the ToDO list, as is the custom definition of namespaces.

Regarding your requirements, point by point:

  • RDF4J is open source (EDL license)
  • It works with Wikidata or any other SPARQL endpoint
  • Works on Linux or any other OS (Java)
  • Does not (yet) support handling namespace prefixes without having to specify them (at least not for SPARQL endpoints)
  • You don't have to write the request in a text file, but you can if you want

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.