Setup Carbon Ldp With Stardog Container From Scratch

Posted on Jul 17, 2018 (last modified Jun 1, 2021)

In this post, I provide a procedure for setting up Carbon LDP with Stardog as the database, from scratch. Further documentation can be found on the product documentation page, Enterprise-class repositories. This procedure will detail how to run both Carbon LDP and Stardog together as two linked Docker containers.

Obtain Stardog and license key file

To create the Stardog container, you will need the Stardog binary and a license key. Head over to stardog.com and fetch yourself the 30-day Enterprise trial or Community Edition.

Setup the Stardog docker container

First, we’ll setup the Stardog container. For this, we’ll use Jon Tutcher’s Stardog dockerfile. Use the following command to get Jon’s repo. In my case, I am providing a new name for the local repo (stardog-docker-jontutcher).

git clone https://github.com/jontutcher/stardog-docker.git stardog-docker-jontutcher

Navigate into the directory and create a resources directory inside of it…

cd stardog-docker-jontutcher/
mkdir resources

Put the Stardog license key file into the resources directory. The file must be named stardog-license-key.bin. You can obtain a license key for the 30 day enterprise trial or community edition at stardog.com. The license key will be picked up from the resources directory and built into the image when we build from the dockerfile.

Back out of the resources directory and run the docker build command.

cd ..
docker build --tag stardog:latest .

Now use the following command to run the container.

docker run \\
    -d -p 5820:5820 \\
    --name stardog-instance \\
    -v /Users/cburleson/data/stardog:/stardog \\
    stardog:latest

Note that the -v parameter should point to your Stardog data directory on your host machine. If you do not have an existing Stardog data directory, you should create an empty one and refer to that.

You can use the following command to verify the container exists and is running:

docker ps -a

In your web browser, navigate to http://localhost:5820/ to access the Stardog administrative interface. You can login with the default username and password: admin | admin. Alternatively, you can access Stardog through Stardog Studio, a separate download.

Create a database in Stardog for Carbon LDP

Next, we’ll create a database in the Stardog server for the Carbon platform to use. Through the web interface, you can do this through Databases > New DB. Create a new database with all the default options except for “Database online”, which you should switch to OFF. This will allow us to edit the database configuration directly after creating it. Name the database whatever you want, such as “carbonldp”. In my case, I’m using the name “carbonldp-industrial” for the industrial PoC I’m building.

After creating the database, edit it and set Query all graphs to ON, which is necessary for Carbon to function properly.

Now Save the database configuration and turn the database ON.

Configure and run the Carbon LDP platform container

Spinning up a Carbon LDP platform container requires only one docker command. But first, you need to…

Create a persistent volume for Carbon LDP

Create a directory on your local machine that can be used as a persistent volume for the Carbon platform container. In my case, I created the following directory, for example:

/Users/cburleson/data/carbonldp-industrial

Run the Carbon LDP container

The prerequisites are set. So, now we can spin up a Carbon server with the following Docker command. Be sure to replace the -v parameter with the path you created for your persistent Carbon directory. Leave everything after the colon in tact (don’t touch the part, :/opt/carbonldp/shared). At the time of this writing, the latest version of Carbon is 1.0.x-alpha. If there is a later version by the time you read this, you can use that. Also be sure to change the name of the carbonldp.repository.id parameter to the name of the database you created for Carbon in the Stardog server. You can name the container whatever you want. It’s typical to call it “carbonldp”, but in my case, I’m running it with the container named “carbonldp-industrial” This allows me to have multiple carbon containers for different purposes. As per my own settings, the full command is as follows:

docker run -d --name carbonldp-industrial -p 8083:8083 \\
    -v /Users/cburleson/data/carbonldp-industrial:/opt/carbonldp/shared \\
    --link stardog-instance:stardog-instance carbonldp/carbonldp-platform:1.0.x-alpha \\
    --carbonldp.repository.type="stardog" \\
    --carbonldp.repository.url="http://stardog-instance:5820/" \\
    --carbonldp.repository.username="admin" \\
    --carbonldp.repository.password="admin" \\
    --carbonldp.repository.id="carbonldp-industrial"

In order for the two containers to communicate with one another, they are linked with Docker link parameter. The value of that parameter is the container (<name or id>:alias or <name or id>). In my case, stardog-instance:stardog-instance, where stardog-instance is the name of the Stardog container and also referred to in the carbonldp.repository.url value.

If the Carbon image is not yet present on your machine, it will be fetched by the command from Docker Hub. You can execute the following Docker command to verify that the container is up and running.

docker ps -a

Is the STATUS is Exited, there was a problem in the startup. If you see such a problem, you can run the following command to examine the logs for troubleshooting:

docker logs <container-name>

Or, in my case…

docker logs carbonldp-industrial

If Carbon started up successfully, its logs should resemble the following:

├┬---------------------------------------------------------------------------------------------------┤
 ¦ Platform: v1.0.0-alpha.13 (Build: 2018-04-12T16:39:27+0000)
 ¦
 ¦ https://carbonldp.com
└┴---------------------------------------------------------------------------------------------------┘

INFO  [                         c.c.Application] -- Starting Application v1.0.0-alpha.13 on 6a8195e0e2e2 with PID 7 (/opt/carbonldp/carbonldp-platform.jar started by root in /)
DEBUG [                         c.c.Application] -- Running with Spring Boot v2.0.1.RELEASE, Spring v5.0.5.RELEASE
INFO  [                         c.c.Application] -- The following profiles are active: default
DEBUG [c.c.r.t.TxnConfig$$EnhancerBySpringCGLIB$$5dca10a3] -- Connecting with configured Stardog repository...
DEBUG [c.c.r.t.TxnConfig$$EnhancerBySpringCGLIB$$5dca10a3] -- Connecting with configured Stardog repository...DONE
DEBUG [                     c.c.l.LockingConfig] -- Starting embedded ZooKeeper server...
DEBUG [                     c.c.l.LockingConfig] -- Starting embedded ZooKeeper server...COMPLETE
DEBUG [   c.c.s.a.t.JWTokenAuthenticationFilter] -- Initializing filter 'jwTokenAuthenticationFilter'
DEBUG [   c.c.s.a.t.JWTokenAuthenticationFilter] -- Filter 'jwTokenAuthenticationFilter' configured successfully
DEBUG [ c.c.s.a.a.AnonymousAuthenticationFilter] -- Initializing filter 'anonymousAuthenticationFilter'
DEBUG [ c.c.s.a.a.AnonymousAuthenticationFilter] -- Filter 'anonymousAuthenticationFilter' configured successfully
INFO  [                         c.c.Application] -- Started Application in 11.816 seconds (JVM running for 13.061)

Verify Carbon data in the Stardog server

For an additional validation, you can verify that the database in Stardog has some preliminary Carbon-specific data in it. In the Stardog web console at http://localhost:5820/, navigate to the database you created for Carbon, click on >_ Query and run the following SPARQL query for Types:

select distinct ?type ?label 
where { 
  ?s a ?type . 
  OPTIONAL { ?type rdfs:label ?label } 
}

You should see Carbon-specific types in the results as shown below…

Verify Carbon response in web browser

You can also do a check for a response from the Carbon server by hitting the following URL in your web browser:

http://localhost:8083/.system/platform/

The browser should return an RDF response similar to the following (the platform meta, which includes the build date and version):

<rdf:RDF xmlns:rdf="http://www.w3.org/1999/02/22-rdf-syntax-ns#">
  <rdf:Description rdf:about="http://localhost:8083/.system/platform/">
    <rdf:type rdf:resource="http://www.w3.org/ns/ldp#Container"/>
    <rdf:type rdf:resource="https://carbonldp.com/ns/v1/platform#RequiredSystemDocument"/>
    <rdf:type rdf:resource="http://www.w3.org/ns/ldp#RDFSource"/>
    <rdf:type rdf:resource="http://www.w3.org/ns/ldp#BasicContainer"/>
    <rdf:type rdf:resource="http://www.w3.org/ns/ldp#Resource"/>
    <rdf:type rdf:resource="https://carbonldp.com/ns/v1/platform#Document"/>
    <rdf:type rdf:resource="https://carbonldp.com/ns/v1/platform#Platform"/>
    <membershipResource xmlns="http://www.w3.org/ns/ldp#" rdf:resource="http://localhost:8083/.system/platform/"/>
    <modified xmlns="https://carbonldp.com/ns/v1/platform#" rdf:datatype="http://www.w3.org/2001/XMLSchema#dateTime">2018-07-17T20:06:25.656Z</modified>
    <created xmlns="https://carbonldp.com/ns/v1/platform#" rdf:datatype="http://www.w3.org/2001/XMLSchema#dateTime">2018-07-17T20:06:25.656Z</created>
  </rdf:Description>
  <rdf:Description rdf:nodeID="node1ciktjenqx53">
    <rdf:type rdf:resource="https://carbonldp.com/ns/v1/platform#PlatformInstance"/>
    <rdf:type rdf:resource="https://carbonldp.com/ns/v1/platform#VolatileResource"/>
    <buildDate xmlns="https://carbonldp.com/ns/v1/platform#" rdf:datatype="http://www.w3.org/2001/XMLSchema#dateTime">2018-04-12T16:39:27.000Z</buildDate>
    <version xmlns="https://carbonldp.com/ns/v1/platform#">1.0.0-alpha.13</version>
  </rdf:Description>
    <rdf:Description rdf:about="http://localhost:8083/.system/platform/">
    <instance xmlns="https://carbonldp.com/ns/v1/platform#" rdf:nodeID="node1ciktjenqx53"/>
  </rdf:Description>
</rdf:RDF>

You’re good to go! You may, however, wish to use the Carbon LDP Workbench as well.

Run the Carbon LDP Workbench

The Carbon LDP Workbench provides a visual interface to your Carbon instance data. You can spin up the workbench with the following command:

docker run -d --name carbonldp-workbench -p 8000:80 \\
    -e "CARBON_HOST=localhost:8083" \\
    -e "CARBON_PROTOCOL=http" \\
    carbonldp/carbonldp-workbench:1.0.x-alpha

You can then access the Workbench at the following URL:

http://localhost:8000/

If you see any errors in the Workbench when first accessing it, try a hard refresh in your browser.

Accessing the Stardog container’s shell

If you want to get into the Stardog container to execute stardog-admin commands, use the following Docker command:

docker exec -it <stardog-container-name> bash

In my case:

docker exec -it stardog-instance bash

Then you can change into the stardog/bin directory…

cd /stardog/bin

And execute stardog-admin commands, such as…

./stardog-admin help

You can type exit to exit out of the container and back to your host shell.

Accessing the Carbon LDP container’s shell

If you want to get into the Carbon LDP container, you can use a similar command. However, the Carbon container does not include bash, so you can use sh instead…

docker exec -it <carbon-container-name> sh

You can type exit to exit out of the container and back to your host shell.

Conclusion

This post described the procedure for setting up a Carbon LDP container configured to run with a Stardog database in a Stardog container. Additionally, we setup the Carbon LDP Workbench. And finally, I showed how you can get inside the Stardog container to execute stardog-admin commands. The procedure results in three Docker containers providing everything you need for Carbon LDP with Stardog development environment.