Apache DataFu is available for download as a source release and as compiled artifacts stored in a Maven repository. Please note that the latest version of datafu-pig and datafu-hourglass is not the same as the latest version of datafu-spark.
The latest source release can be found here:
It is important to validate the release using either the PGP signature (
.asc file) or hashes (
.sha512 file). For more information on verification of Apache releases, see here. The
KEYS file can be found here.
Once fetched, the
KEYS file can be imported and the
.asc file can used to verify the release.
gpg --import KEYS gpg --verify apache-datafu-sources-1.7.0.tgz.asc apache-datafu-sources-1.7.0.tgz
sha512sum tool can be used to compute a SHA-512 hash that can be compared against the
sha512sum apache-datafu-sources-1.7.0.tgz cat apache-datafu-sources-1.7.0.tgz.sha512
Note that the hashes are only intended to check that the file has been downloaded correctly. They do not provide guarantees on the authenticity of the file. The signature should instead be used for this purpose. For more information see here.
Make sure you have Gradle installed. Extract the source and bootstrap the
gradlew script that's used for building. The
gradlew script uses the specific version of Gradle that DataFu is intended to be built with.
tar xvf apache-datafu-sources-1.7.0.tgz cd apache-datafu-sources-1.7.0 gradle -b bootstrap.gradle
To build the JARs, run:
This will produce JARs in the following directories:
DataFu artifacts can be installed to your local maven repository like so:
Assuming your local maven repository is at
~/.m2, you should see the DataFu artifacts under
~/.m2/repository/org/apache/datafu/. You should now be able to declare a dependency on DataFu artifacts as shown below.
The latest release can be found in Apache's Maven Repository for DataFu:
You can also use a dependency management system to download the DataFu artifacts and all their dependencies. Some examples appear below.
libraryDependencies += "org.apache.datafu" %% "datafu-spark" % "1.7.0" intransitive
<dependency> <groupId>org.apache.datafu</groupId> <artifactId>datafu-spark</artifactId> <version>1.7.0</version> </dependency>
See the following guides for next steps: