# Install hadoop with Homebrew, chocolatey, MacPorts, Nix Framework for distributed processing of large data sets. Version 3.5.0 via Homebrew; verified 2026-06-22. ## Install ```sh sudo av install brew:hadoop ``` Additional install commands: ### macOS - Homebrew (100%): ```sh brew install hadoop ``` Evidence: local Homebrew formula metadata - MacPorts (94%): ```sh sudo port install hadoop ``` Evidence: MacPorts ports tree: java/hadoop/Portfile from https://api.github.com/repos/macports/macports-ports/git/trees/master?recursive=1 ### Linux - Nix (92%): ```sh nix profile install nixpkgs#hadoop ``` Evidence: nixpkgs package indexes: hadoop from https://raw.githubusercontent.com/NixOS/nixpkgs/master/pkgs/top-level/all-packages.nix ### Windows - Chocolatey (92%): ```sh choco install hadoop ``` Evidence: Chocolatey community package catalog: hadoop from http://community.chocolatey.org/api/v2/Packages?$filter=IsLatestVersion&$select=Id&$top=1000&$skiptoken='8.884','vim-x64' ## Package facts - **Package key:** brew:hadoop - **Package manager:** Homebrew - **Package manager page:** - **Version:** 3.5.0 - **Source summary:** Framework for distributed processing of large data sets - **Homepage:** - **Repository:** - **Upstream docs:** - **License:** Apache-2.0 - **Source archive:** - **Last updated:** 2026-06-22T14:03:41-07:00 - **Generated:** 2026-07-08T07:18:31+00:00 ## Executables - FederationStateStore (cli) - distribute-exclude.sh (cli) - hadoop (cli) - hadoop-daemon.sh (cli) - hadoop-daemons.sh (cli) - hdfs (cli) - httpfs.sh (cli) - kms.sh (cli) - mapred (cli) - mr-jobhistory-daemon.sh (cli) - refresh-namenodes.sh (cli) - start-all.sh (cli) - start-balancer.sh (cli) - start-dfs.sh (cli) - start-secure-dns.sh (cli) - start-yarn.sh (cli) - stop-all.sh (cli) - stop-balancer.sh (cli) - stop-dfs.sh (cli) - stop-secure-dns.sh (cli) - stop-yarn.sh (cli) - workers.sh (cli) - yarn-daemon.sh (cli) - yarn-daemons.sh (cli) - FederationStateStore (alias) - distribute-exclude.sh (alias) - hadoop (alias) - hadoop-daemon.sh (alias) - hadoop-daemons.sh (alias) - hdfs (alias) - httpfs.sh (alias) - kms.sh (alias) - mapred (alias) - mr-jobhistory-daemon.sh (alias) - refresh-namenodes.sh (alias) - start-all.sh (alias) - start-balancer.sh (alias) - start-dfs.sh (alias) - start-secure-dns.sh (alias) - start-yarn.sh (alias) - stop-all.sh (alias) - stop-balancer.sh (alias) - stop-dfs.sh (alias) - stop-secure-dns.sh (alias) - stop-yarn.sh (alias) - workers.sh (alias) - yarn-daemon.sh (alias) - yarn-daemons.sh (alias) ## Dependencies - openjdk@17 ## Install behavior - Post-install hook: not defined - Bottle: available on all ## Freshness - Page generated: 2026-07-08 - Package-manager version: 3.5.0 - Package-manager updated: 2026-06-22 - Local data: ok - Upstream repository: https://hadoop.apache.org/ - info: Release/tag comparison is only available for GitHub repositories. ## Project history and usage Apache Hadoop is the Apache Software Foundation framework for reliable, scalable, distributed computing across clusters. The Homebrew package exposes the command-line surface around Hadoop Common, HDFS, YARN, and MapReduce, making it a developer workstation entry point into a historically server-oriented big-data stack. ### Project history Apache's early Hadoop documentation records the project split from Nutch MapReduce and distributed filesystem code in 2006, with Hadoop code moved into its own source tree in February, project approval in March, and release 0.1.0 in April. The project grew around a practical abstraction: store data across many machines with HDFS and run distributed computation with MapReduce. Hadoop 1.0.0 arrived in December 2011 after what Apache called six years of gestation. That release came from the 0.20-security line and included security, HBase-related HDFS append/sync work, WebHDFS, and performance improvements. The 1.x era established Hadoop as a deployable foundation for large-scale storage and batch processing. The 2.x line changed Hadoop's architecture by introducing YARN, described in Apache release notes as NextGen MapReduce. YARN separated cluster resource management from the MapReduce programming model, letting Hadoop run a wider set of distributed applications while keeping HDFS as a common storage layer. Apache's 2.0.2-alpha notes also recorded YARN deployment on a 2000-node cluster at release time. Hadoop 3.x continued the infrastructure evolution. The 3.0.0 documentation highlights erasure coding in HDFS for lower storage overhead than three-way replication, along with a broader set of HDFS, YARN, MapReduce, and operational features. Later Apache site news documents 3.4 and 3.5 lines as large maintenance and enhancement releases. ### Adoption history Hadoop's adoption history is tied to the rise of commodity-cluster data processing. The Apache homepage describes a framework designed to scale from single servers to thousands of machines, with failure detection and handling in the application layer rather than relying only on hardware high availability. The broader ecosystem made Hadoop more than one binary. Apache release notes and docs refer to HBase, Pig, Oozie, Hive, Sqoop, WebHDFS, and YARN applications as downstream or companion systems. For package managers, that ecosystem means Hadoop often arrives as a dependency, local test harness, or compatibility target even when production clusters are provisioned by other means. ### How it is used On a workstation, the package is commonly used for command-line tools such as `hadoop`, `hdfs`, `yarn`, and `mapred`, for local pseudo-distributed testing, for interacting with HDFS-compatible services, or for building software that expects Hadoop client libraries and scripts. The single-node setup docs are a common starting point because they let developers run Hadoop locally before touching a cluster. Operational Hadoop usage is configuration-heavy. Apache cluster setup documentation names site-specific XML files under `etc/hadoop`, including core-site.xml, hdfs-site.xml, yarn-site.xml, and mapred-site.xml, plus environment scripts such as hadoop-env.sh and yarn-env.sh. The same docs instruct administrators to distribute configuration to HADOOP_CONF_DIR across machines. ### Why package nerds care Hadoop is a heavyweight formula by package-manager standards: a Java-centric distributed-systems stack with many scripts, XML configs, daemon roles, and ecosystem expectations. It matters to package nerds because installing it locally can satisfy client tooling, build-time assumptions, demos, and compatibility tests without standing up a full distribution. It is also a naming and dependency landmark. Many big-data packages historically needed to say which Hadoop line, HDFS client, YARN API, or MapReduce compatibility level they targeted. That makes the package less about a single executable and more about a versioned contract with the surrounding data ecosystem. ### Timeline - 2006: Hadoop code moved out of Nutch, the project was approved, and release 0.1.0 was published. - 2011: Hadoop 1.0.0 was released from the 0.20-security code line. - 2012: Hadoop 2.0.0-alpha introduced HDFS HA, YARN, HDFS Federation, and protobuf wire compatibility. - 2012: Hadoop 2.0.2-alpha notes recorded a more stable YARN deployed on a 2000-node cluster. - 2017: Hadoop 3.0.0 documentation described HDFS erasure coding and other 3.x architecture updates. - 2024: Apache announced the Hadoop 3.4 line with thousands of fixes, improvements, and enhancements since 3.3. - 2026: Apache announced the Hadoop 3.5 line with hundreds of fixes, improvements, and enhancements since 3.4. ### Related projects - Apache Nutch: origin project from which Hadoop's early MapReduce and distributed filesystem code split. - HDFS: Hadoop Distributed File System storage layer. - YARN: Hadoop cluster resource-management framework. - MapReduce: batch-processing framework and command surface in Hadoop. - Apache HBase, Hive, Pig, Oozie, and Sqoop: common Hadoop-era ecosystem projects referenced in Apache release and setup material. ### Sources - - - - - - - - ## Security Notes broad file, network, media, or database tool signal. - **Geiger risk:** blue / medium - broad file, network, media, or database tool signal ## Configuration and credential file locations These source-backed paths show where this package keeps local settings or durable credentials. Automic Vault can use them as review targets for secret scanning, migration, and command approval. ## Configuration files - Unix: $HADOOP_CONF_DIR/core-site.xml, $HADOOP_CONF_DIR/hdfs-site.xml, $HADOOP_CONF_DIR/mapred-site.xml, $HADOOP_CONF_DIR/yarn-site.xml, etc/hadoop/core-site.xml, etc/hadoop/hdfs-site.xml, etc/hadoop/mapred-site.xml, etc/hadoop/yarn-site.xml ## Source Database Details - **Source Database:** Homebrew formula API - **Tap:** homebrew/core - **Full Name:** hadoop - **Version Scheme:** 0 - **Revision:** 0 - **Conflicts With:** corepack, yarn - **Bottle Stable Root URL:** - **Deprecated:** no - **Disabled:** no - **Keg Only:** no - **URL Keys:** stable ## Other Package-Manager Records - Nix - hadoop: normalized package name match | nixpkgs package indexes: hadoop from https://raw.githubusercontent.com/NixOS/nixpkgs/master/pkgs/top-level/all-packages.nix - MacPorts - hadoop: normalized package name match | MacPorts ports tree: java/hadoop/Portfile from https://api.github.com/repos/macports/macports-ports/git/trees/master?recursive=1 - Chocolatey - hadoop: normalized package name match | Chocolatey community package catalog: hadoop from http://community.chocolatey.org/api/v2/Packages?$filter=IsLatestVersion&$select=Id&$top=1000&$skiptoken='8.884','vim-x64' ## Related links - [Secret-risk packages](https://www.automicvault.com/pkg/secret-risk-packages/) - Has protected-tool coverage, approval-gate, or non-low Geiger security signals. - [Terminal utility packages](https://www.automicvault.com/pkg/terminal-utilities/) - Matched terminal and command-line workflow metadata. - [Text processing packages](https://www.automicvault.com/pkg/text-processing-tools/) - Matched text, document, or structured-data processing metadata. - [Networking and protocol packages](https://www.automicvault.com/pkg/networking-protocol-tools/) - Matched network, protocol, or remote-service metadata. - [openjdk@17](https://www.automicvault.com/pkg/brew/openjdk-17/) - Runtime dependency declared by Homebrew. - [flume](https://www.automicvault.com/pkg/brew/flume/) - Popular package that depends on this formula. - [hive](https://www.automicvault.com/pkg/brew/hive/) - Popular package that depends on this formula. - [storm](https://www.automicvault.com/pkg/brew/storm/) - Shares av.db curated category or tags: cli, data, distributed-computing. - [pig](https://www.automicvault.com/pkg/brew/pig/) - Shares av.db curated category or tags: big-data, cli, data, mapreduce. - [apache-drill](https://www.automicvault.com/pkg/brew/apache-drill/) - Shares av.db curated category or tags: big-data, cli, data. - [hbase](https://www.automicvault.com/pkg/brew/hbase/) - Shares av.db curated category or tags: big-data, cli, data. - [libgrape-lite](https://www.automicvault.com/pkg/brew/libgrape-lite/) - Shares av.db curated category or tags: cli, data, distributed-computing. - [prestodb](https://www.automicvault.com/pkg/brew/prestodb/) - Shares av.db curated category or tags: big-data, cli, data. - [trino](https://www.automicvault.com/pkg/brew/trino/) - Shares av.db curated category or tags: big-data, cli, data. - [cassandra](https://www.automicvault.com/pkg/brew/cassandra/) - Local package facts share a topical domain. Shared terms: cli, data, distributed, openjdk, openjdk-17. ## Combined YAML source View the package source record on GitHub. [combined/hadoop.yml](https://github.com/automic-vault/db/blob/main/combined/hadoop.yml) ## Sources - Nucleus package database - Geiger risk classifier - package-page enrichment - curated configuration and credential file locations - curated package history - package version freshness - av.db category and tag curation - package relationship graph - external package-manager database matches - cross-ecosystem install command graph