1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
|
= Living space
:description: Documentation for the Living space
:sectanchors:
:url-repo: https://git.tammi.cc/housing.git
SVG and WGS84 https://www.w3.org/TR/SVG11/coords.html
The project consists of three parts:
1. Application (native web browser)
2. CouchDB and S3
3. Scraper (Golang)
The developments tools are listed in flake.nix. The development environment can be start with command:
[source,bash]
----
nix develop
----
== Application
There is a development server `server.js` that serves the app and static assets.
The application resides in app directory.
[%header,format=tsv]
|===
include::./app/requirements.tsv[]
|===
Run (the development server) with:
[source,bash]
----
node server.js
----
Static assets are deployed directly to a web server with Git.
Later it could be possible to download the assets to CounchDB.
== Scraper runner
Golang implementation fetches the data to CouchDB.
The runner resides in scrape directory.
[%header,format=tsv]
|===
include::./scrape/requirements.tsv[]
|===
The running will require some cookies to be gotten from a web browser. Run with:
[source,bash]
----
export OTA_TOKEN=...
export OTA_CUID=...
export OTA_LOADED=...
export PHPSESSID=...
go run main.go
----
== Next steps
- Implement additional map features, "metro", "pikaraitiotie", "koulut", "kaupat".
- Add possibility to color districts based on data such as crime rate, etc.
- Make the weight calculation work.
- Fix the map zoom and initial viewport
- Add links to house details on the service. Add additional images
- Parse more data from data source. Currently only overview is parsed.
- Images on modal open on click to a new window
- Visual programming? Value function description with Javascript?
- Notifications to user on new houses
- Sharing via URL
- Real support for MultiLineString in geometry
== Analysis Data processing
List of open WFS data sources are in:
https://kartta.hel.fi/avoindata/dokumentit/Aineistolista_wfs_avoindata.html
WFS Capabilities can be found from:
https://kartta.hel.fi/ws/geoserver/avoindata/wfs?version=2.0.0&request=GetCapabilities
The data can then be downloaded with CURL. Examples below:
=== Roads
[source,bash]
----
# First download the full dataset
curl -X GET \
"https://kartta.hel.fi/ws/geoserver/avoindata/wfs?\
service=WFS&\
version=2.0.0&\
request=GetFeature&\
typeNames=avoindata:Liikennevaylat&\
outputFormat=application/json" \
-o liikennevaylat_full.geojson
# Then filter with jq
jq '{
type: .type,
features: [.features[] | select(.properties.alatyyppi == "Moottoriväylä")],
crs: .crs
}' liikennevaylat_full.geojson > moottorivaylat.geojson
----
[source,bash]
----
jq '[.features[].properties.paatyyppi | select(. != null)] | unique' liikennevaylat_full.geojson
----
=== Train stations
[source,bash]
----
curl -X GET \
"https://kartta.hel.fi/ws/geoserver/avoindata/wfs?\
service=WFS&\
version=2.0.0&\
request=GetFeature&\
typeNames=avoindata:Seutukartta_liikenne_juna_asema&\
outputFormat=application/json&\
srsName=EPSG:4326" \
-o juna_asemat.geojson
----
=== Railway lines
[source,bash]
----
curl -X GET \
"https://kartta.hel.fi/ws/geoserver/avoindata/wfs?\
service=WFS&\
version=2.0.0&\
request=GetFeature&\
typeNames=avoindata:Seutukartta_liikenne_juna_rata&\
outputFormat=application/json&\
srsName=EPSG:4326" \
-o juna_radat.geojson
----
|
