Skip to content

Commit 36c5e01

Browse files
committed
Avoid failure when public schema already exists. Minor docs improvement
1 parent d23ba85 commit 36c5e01

File tree

3 files changed

+39
-5
lines changed

3 files changed

+39
-5
lines changed

README.md

Lines changed: 22 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -38,9 +38,8 @@ The easiest option to use PgOSM-Flex is with the
3838
The image has all the pre-reqs installed, handles downloading an OSM subregion
3939
from Geofabrik, and saves an output `.sql` file with the processed data
4040
ready to load into your database(s).
41-
42-
The script in the Docker image uses `PGOSM_DATE` to enable the Docker process
43-
to archive source PBF files and easily reload them at a later date.
41+
The PBF/MD5 source files are archived by date with the ability to
42+
easily reload them at a later date.
4443

4544

4645
### Basic Docker usage
@@ -97,18 +96,36 @@ docker exec -it \
9796
--subregion=district-of-columbia
9897
```
9998

100-
10199
The initial output with the `docker exec` command points to the log file
102100
(linked in the `docker run` command above), monitor this file to track
103101
progress of the import.
104102

105-
106103
```bash
107104
Monitor /app/output/district-of-columbia.log for progress...
108105
If paths setup as outlined in README.md, use:
109106
tail -f ~/pgosm-data/district-of-columbia.log
110107
```
111108

109+
The above command takes roughly 1 minute to run if the PBF for today
110+
has already been downloaded.
111+
If the PBF is not downloaded it will depend on how long
112+
it takes to download the 17 MB PBF file + ~ 1 minute processing.
113+
114+
The output `.sql` file is saved under
115+
`~/pgosm_data/pgosm-flex-north-america-us-district-of-columbia-run-all.sql`.
116+
This `.sql` file can be loaded into a PostGIS enabled database
117+
using:
118+
119+
```bash
120+
psql -d postgres -c "CREATE DATABASE myosm;"
121+
psql -d myosm -c "CREATE EXTENSION postgis;"
122+
```
123+
124+
```bash
125+
psql -d myosm \
126+
-f ~/pgosm-data/pgosm-flex-north-america-us-district-of-columbia-run-all.sql
127+
```
128+
112129

113130
### After processing
114131

docker/db.py

Lines changed: 16 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -6,6 +6,8 @@
66
import subprocess
77

88
import psycopg2
9+
import sh
10+
911

1012
LOGGER = logging.getLogger('pgosm-flex')
1113

@@ -362,3 +364,17 @@ def run_pg_dump(export_filename, out_path, data_only, schema_name):
362364
check=False)
363365
LOGGER.info(f'pg_dump complete, saved to {export_path}')
364366
LOGGER.debug(f'pg_dump output: \n {output.stderr}')
367+
fix_pg_dump_create_public(export_path)
368+
369+
370+
def fix_pg_dump_create_public(export_path):
371+
"""Using pg_dump with `--schema=public` results in
372+
a .sql script containing `CREATE SCHEMA public;`, nearly always breaks
373+
in target DB. Replaces with `CREATE SCHEMA IF NOT EXISTS public;`
374+
"""
375+
result = sh.sed('-i',
376+
's/CREATE SCHEMA public;/CREATE SCHEMA IF NOT EXISTS public;/',
377+
export_path)
378+
LOGGER.debug('Completed replacement to not fail when public schema exists')
379+
LOGGER.debug(result)
380+

requirements.txt

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -2,3 +2,4 @@ click>=8.0.1
22
coverage>=5.5
33
psycopg2>=2.8.6
44
requests>=2.26.0
5+
sh>=1.14.2

0 commit comments

Comments
 (0)