Skip to content

Commit ba5b184

Browse files
committed
New doc version references
1 parent 7a51650 commit ba5b184

8 files changed

Lines changed: 20 additions & 24 deletions

File tree

‎docs/bigdata/README.md‎

Lines changed: 6 additions & 8 deletions
Original file line numberDiff line numberDiff line change
@@ -1,4 +1,4 @@
1-
# Disaggregated HDP Spark and Hive with Obstor
1+
# Disaggregated Spark and Hive with Obstor
22

33
## 1. Cloud-native Architecture
44

@@ -12,9 +12,7 @@ Obstor also supports multi-cluster, multi-site federation similar to AWS regions
1212

1313
## 2. Prerequisites
1414

15-
* Install Hortonworks Distribution using this [guide.](https://docs.hortonworks.com/HDPDocuments/Ambari-2.7.1.0/bk_ambari-installation/content/ch_Installing_Ambari.html)
16-
* [Setup Ambari](https://docs.hortonworks.com/HDPDocuments/Ambari-2.7.1.0/bk_ambari-installation/content/set_up_the_ambari_server.html) which automatically sets up YARN
17-
* [Installing Spark](https://docs.hortonworks.com/HDPDocuments/HDP3/HDP-3.0.1/installing-spark/content/installing_spark.html)
15+
* Install a Hadoop cluster with Spark and Hive. Follow the [Apache Hadoop](https://hadoop.apache.org/docs/stable/) cluster setup (which configures YARN) and the [Apache Spark](https://spark.apache.org/docs/latest/) installation guides.
1816
* Install Obstor Distributed Server using one of the guides below.
1917
* [Deployment based on Kubernetes](/docs/orchestration/kubernetes)
2018

@@ -95,7 +93,7 @@ fs.s3a.threads.max=2048 # Maximum number of threads for S3A
9593
The rest of the other optimization options are discussed in the links below
9694

9795
* [https://hadoop.apache.org/docs/current/hadoop-aws/tools/hadoop-aws/index.html](https://hadoop.apache.org/docs/current/hadoop-aws/tools/hadoop-aws/index.html)
98-
* [https://hadoop.apache.org/docs/r3.1.1/hadoop-aws/tools/hadoop-aws/committers.html](https://hadoop.apache.org/docs/r3.1.1/hadoop-aws/tools/hadoop-aws/committers.html)
96+
* [https://hadoop.apache.org/docs/current/hadoop-aws/tools/hadoop-aws/committers.html](https://hadoop.apache.org/docs/current/hadoop-aws/tools/hadoop-aws/committers.html)
9997

10098
Once the config changes are applied, proceed to restart **Hadoop** services.
10199

@@ -168,7 +166,7 @@ hive.mv.files.threads=40
168166
mapreduce.input.fileinputformat.list-status.num-threads=50
169167
```
170168

171-
For more information about these options please visit [https://www.cloudera.com/documentation/enterprise/5-11-x/topics/admin_hive_on_s3_tuning.html](https://www.cloudera.com/documentation/enterprise/5-11-x/topics/admin_hive_on_s3_tuning.html)
169+
For more information about these options please visit [https://hadoop.apache.org/docs/current/hadoop-aws/tools/hadoop-aws/performance.html](https://hadoop.apache.org/docs/current/hadoop-aws/tools/hadoop-aws/performance.html)
172170

173171
![hive-config](https://raw.githubusercontent.com/cloudment/obstor/main/docs/bigdata/images/image13.png "hive advanced custom config")
174172

@@ -191,7 +189,7 @@ Follow these steps to run the Spark Pi example:
191189
* Navigate to a node with the Spark client and access the spark2-client directory:
192190

193191
```bash
194-
cd /usr/hdp/current/spark2-client
192+
cd $SPARK_HOME
195193
su spark
196194
```
197195

@@ -227,7 +225,7 @@ The following example submits WordCount code to the Scala shell. Select an input
227225
* Navigate to a node with Spark client and access the spark2-client directory:
228226

229227
```bash
230-
cd /usr/hdp/current/spark2-client
228+
cd $SPARK_HOME
231229
su spark
232230
```
233231

‎docs/bucket/notifications/README.md‎

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -870,7 +870,7 @@ The steps below show how to use this notification target in `namespace` format.
870870
871871
### Step 1: Ensure minimum requirements are met
872872
873-
Obstor requires PostgreSQL version 9.5 or above. Obstor uses the [`INSERT ON CONFLICT`](https://www.postgresql.org/docs/9.5/static/sql-insert.html#SQL-ON-CONFLICT) (aka UPSERT) feature, introduced in version 9.5 and the [JSONB](https://www.postgresql.org/docs/9.4/static/datatype-json.html) data-type introduced in version 9.4.
873+
Obstor requires PostgreSQL version 17 or above. Obstor uses the [`INSERT ON CONFLICT`](https://www.postgresql.org/docs/9.5/static/sql-insert.html#SQL-ON-CONFLICT) (aka UPSERT) feature, introduced in version 9.5 and the [JSONB](https://www.postgresql.org/docs/9.4/static/datatype-json.html) data-type introduced in version 9.4.
874874
875875
### Step 2: Add PostgreSQL endpoint to Obstor
876876
@@ -1004,7 +1004,7 @@ The steps below show how to use this notification target in `namespace` format.
10041004
10051005
### Step 1: Ensure minimum requirements are met
10061006
1007-
Obstor requires MySQL version 5.7.8 or above. Obstor uses the [JSON](https://dev.mysql.com/doc/refman/5.7/en/json.html) data-type introduced in version 5.7.8. We tested this setup on MySQL 5.7.17.
1007+
Obstor requires MySQL version 8.0 or above. Obstor uses the [JSON](https://dev.mysql.com/doc/refman/8.4/en/json.html) data-type introduced in version 5.7.8. We tested this setup on MySQL 8.4.
10081008
10091009
### Step 2: Add MySQL server endpoint configuration to Obstor
10101010

‎docs/integrations/veeam/README.md‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -3,7 +3,7 @@
33
When using Veeam Backup and Replication, you can use S3-compatible object storage such as Obstor as a capacity tier for backups. This disaggregates storage for the Veeam infrastructure and allows you to retain control of your data. With the ease of use of setup and administration of Obstor, it allows a Veeam backup admin to easily deploy their own object store for capacity tiering.
44

55
__Prerequisites__
6-
- One or both of Veeam Backup and Replication with support for S3-compatible object store (e.g. 9.5.4) and Veeam Backup for Office365 (VBO)
6+
- One or both of Veeam Backup and Replication with support for S3-compatible object store (e.g. 12.3) and Veeam Backup for Office365 (VBO)
77
- Obstor object storage set up
88
- Veeam requires TLS connections to the object storage. This can be configured per the [TLS guide](/docs/tls)
99
- The S3 bucket, Access Key and Secret Key have to be created before and outside of Veeam.

‎docs/orchestration/docker-swarm/docker-compose.yaml‎

Lines changed: 1 addition & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -1,8 +1,6 @@
1-
version: '3.7'
2-
31
services:
42
obstor1:
5-
image: ghcr.io/cloudment/obstor:RELEASE.2021-04-22T15-44-28Z
3+
image: ghcr.io/cloudment/obstor
64
hostname: obstor1
75
volumes:
86
- obstor1-data:/export

‎docs/shared-backend/DESIGN.md‎

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -19,7 +19,7 @@ Restrictions
1919

2020
Running Obstor instances on shared backend is no different than running on a stand-alone disk. There are no special configuration changes required to enable this feature. Access to files stored on NAS volume are locked and synchronized by default. Following examples will clarify this further for each operating system of your choice:
2121

22-
### Ubuntu 16.04 LTS
22+
### Ubuntu 26.04 LTS
2323

2424
Example 1: Start Obstor instance on a shared backend mounted and available at `/path/to/nfs-volume`.
2525

@@ -33,7 +33,7 @@ On linux server2
3333
obstor backend nas /path/to/nfs-volume
3434
```
3535

36-
### Windows 2012 Server
36+
### Windows Server 2025
3737

3838
Example 1: Start Obstor instance on a shared backend mounted and available at `\\remote-server\cifs`.
3939

‎docs/shared-backend/README.md‎

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -29,7 +29,7 @@ To run Obstor shared backend instances, you need to start multiple Obstor server
2929
- All the nodes running shared Obstor need to have same access key and secret key. To achieve this, we export access key and secret key as environment variables on all the nodes before executing Obstor server command.
3030
- The drive paths below are for demonstration purposes only, you need to replace these with the actual drive paths/folders.
3131

32-
#### Obstor shared mode on Ubuntu 16.04 LTS.
32+
#### Obstor shared mode on Ubuntu 26.04 LTS.
3333

3434
You'll need the path to the shared volume, e.g. `/path/to/nfs-volume`. Then run the following commands on all the nodes you'd like to launch Obstor.
3535

@@ -39,7 +39,7 @@ export OBSTOR_ROOT_PASSWORD=<SECRET_KEY>
3939
obstor backend nas /path/to/nfs-volume
4040
```
4141

42-
#### Obstor shared mode on Windows 2012 Server
42+
#### Obstor shared mode on Windows Server 2025
4343

4444
You'll need the path to the shared volume, e.g. `\\remote-server\smb`. Then run the following commands on all the nodes you'd like to launch Obstor.
4545

‎docs/sts/etcd.md‎

Lines changed: 5 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -3,20 +3,20 @@ etcd is a distributed key value store that provides a reliable way to store data
33

44
## Get started
55
### 1. Prerequisites
6-
- Docker 18.03 or above, refer here for [installation](https://docs.docker.com/install/).
6+
- Docker 29.5.2 or above, refer here for [installation](https://docs.docker.com/install/).
77

88
### 2. Start etcd
99
etcd uses `gcr.io/etcd-development/etcd` as a primary container registry.
1010

1111
```bash
1212
rm -rf /tmp/etcd-data.tmp && mkdir -p /tmp/etcd-data.tmp && \
13-
docker rmi gcr.io/etcd-development/etcd:v3.3.9 || true && \
13+
docker rmi gcr.io/etcd-development/etcd:v3.6.12 || true && \
1414
docker run \
1515
-p 2379:2379 \
1616
-p 2380:2380 \
1717
--mount type=bind,source=/tmp/etcd-data.tmp,destination=/etcd-data \
18-
--name etcd-gcr-v3.3.9 \
19-
gcr.io/etcd-development/etcd:v3.3.9 \
18+
--name etcd-gcr-v3.6.12 \
19+
gcr.io/etcd-development/etcd:v3.6.12 \
2020
/usr/local/bin/etcd \
2121
--name s1 \
2222
--data-dir /etcd-data \
@@ -29,7 +29,7 @@ rm -rf /tmp/etcd-data.tmp && mkdir -p /tmp/etcd-data.tmp && \
2929
--initial-cluster-state new
3030
```
3131

32-
You may also setup etcd with TLS following this documentation [here](https://coreos.com/etcd/docs/latest/op-guide/security.html)
32+
You may also setup etcd with TLS following this documentation [here](https://etcd.io/docs/v3.6/op-guide/security/)
3333

3434
### 3. Setup Obstor with etcd
3535
Obstor server expects environment variable for etcd as `OBSTOR_ETCD_ENDPOINTS`, this environment variable takes many comma separated entries.

‎docs/sts/wso2.md‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -3,7 +3,7 @@ WSO2 is an Identity Server open source and is released under Apache Software Lic
33

44
## Get started
55
### 1. Prerequisites
6-
- JAVA 1.8 and above installed already and JAVA_HOME points to JAVA 1.8 installation.
6+
- JAVA 25 and above installed already and JAVA_HOME points to JAVA 25 installation.
77
- Download WSO2 follow their [installation guide](https://docs.wso2.com/display/IS540/Installation+Guide).
88

99
### 2. Configure WSO2

0 commit comments

Comments
 (0)