PROJECT 3: Petrol Down Under
Objective: Build an SPA with Google Maps API
Tools: Express, PostgreSQL, Google Maps API, Open Meteo API, Geolocation API, Postcode API
A Higher Calling
This project had three goals: to work with a single-page application (SPA), to work with APIs, and most importantly of all, to work with a team. Together, we had to build a website that would use data from Google Maps to display every single petrol station in Australia for users to see on a map. Thus on this fateful day, TEAM JESUS was born, formed by me and 3 other classmates. The other team in my class was called something fancier like Team Epsilon V but I bet they never screamed their team's name when things didn't just work.
Before we got to anything new, we first had to revisit some old friends: Express and PostgreSQL. Express was there to do its usual job of handling server-side stuff and PostgreSQL once again served as our database manager. We then had to fill our database with over 5000 petrol stations worth of data from a file to get our project started.
const fs = require('fs');
const db = require('./index')
const data = fs.readFileSync('./db/stations.csv',
{ encoding: 'utf8', flag: 'r' });
let lines = data.split('\n')
let brandNamesList = []
async function doDatabase() {
let sql1 = "delete from stations"
let sql2 = "delete from locations"
let sql3 = "delete from owners"
let deleted = await db.query(sql1).then(() => db.query(sql2)).then(() => db.query(sql3))
let owners = await doOwners()
let locations = await doLocations()
let stations = await doStations()
}
async function doOwners() {
for (let line of lines.slice(1)) {
let sections = line.split(',')
let brandName = sections[7]
let sql
let sqlNewOwner = `
INSERT INTO owners
(brand_name)
VALUES ($1)
RETURNING *;
`
let sqlExistingOwner = `
SELECT * FROM owners
WHERE brand_name = $1;
`
if (!brandNamesList.includes(brandName)) {
brandNamesList.push(brandName)
sql = sqlNewOwner
} else {
sql = sqlExistingOwner
}
db.query(sql, [brandName])
.catch(err=>console.log(err))
}
}
async function doLocations() {
for (let line of lines.slice(1)) {
let stationObj = {}
let sections = line.split(',')
stationObj.address = sections[9]
stationObj.suburb = sections[10]
stationObj.lat = sections[sections.length - 2]
stationObj.lng = sections[sections.length - 1]
let sqlLocation = `
INSERT INTO locations
(address, suburb, lat, lng)
VALUES
($1, $2, $3, $4);
`
db.query(sqlLocation, [stationObj.address, stationObj.suburb, stationObj.lat, stationObj.lng])
.catch(err=>console.log(err))
}
}
async function doStations() {
for (let line of lines.slice(1)) {
let stationObj = {}
let sections = line.split(',')
let brandName = sections[7]
let address = sections[9]
stationObj.description = sections[2]
stationObj.name = sections[5]
let sql_ownersId = `SELECT id FROM owners WHERE brand_name = $1`
let sql_locationId = `SELECT id FROM locations WHERE address = $1`
let ownerId = await db.query(sql_ownersId, [brandName]).then(res=>res.rows[0])
let locationId = await db.query(sql_locationId, [address]).then(res=>res.rows[0])
let sqlStation = `
INSERT INTO stations
(owner_id, location_id, station_name, description)
VALUES
($1, $2, $3, $4)
`
db.query(sqlStation, [ownerId.id, locationId.id, stationObj.name, stationObj.description])
.catch(err=>console.log(err))
}
}
doDatabase()
The file we were given was a CSV, basically a file formatted like table only with commas separating data instead of columns.
We read the file using fs.readFileSync and then essentially sliced and spliced the data up to input in our database.
There was one major annoyance that I remember discovering. We split each row of data by commas - as that's how they're meant to be seperated -
but some stations had comments with normal commas inside, meaning we would split the data incorrectly. Thank goodness for Ctrl+F
or we would have to sift through 5000 pieces of data manually to figure it out.
Revelations
With our database populated, we could now move on to the next test of faith: using Google Maps API. This was just one of many APIs, albeit the central one, that we were going to use for this project. All these APIs - Application Programming Interfaces - were basically ways for us to communicate with other software to retrieve some information. In fact, we even technically made our own API with our database. For Google Maps API, we were basically getting all the stuff we needed to make a map like they do.
let map;
async function initMap() {
const { Map } = await google.maps.importLibrary("maps");
map = new Map(document.getElementById("map"), {
center: { lat: -34.397, lng: 150.644 },
zoom: 8,
});
}
initMap();
This is the basic code, provided by Google themselves, to make a generic Google Map. You might have noticed the
async and await terminology in the code being used. While functions normally work line by line,
stopping for each one until its done, async await means that your code won't be held up by things that may take some time (like fetching
data from an API).
Anyway, making a basic map is quite simple. However, this little function would end up causing TEAM JESUS a lot more trouble than expected. More on that later. After this initial set up, we had to split into pairs to do some peer programming for the rest of the assignment. Also more on that later. As a result of this though, it made our code pretty scattered; we were given pre-assigned tasks to accomplish for this project and each feature was split into multiple steps. I could do part 1 of one feature and then only rotate back to it at like part 4 so its a bit hard to run through all the code I did.
Nevertheless, I can at least explain all the other APIs we used. Open Meteo API gave us data about the weather which we used to display the current temperature. Geolocation API was used to find the user's location and move the map to center on said location. Lastly, Postcode API was used to find all the petrol stations under a certain postcode. This last one was interesting/frustrating because unlike the other APIs, we couldn't make a direct request to Postcode API.
// postcode router
const express = require('express')
const router = express.Router()
router.get('/api/postcode/:postcode', (req, res) => {
fetch(`http://v0.postcodeapi.com.au/suburbs/${req.params.postcode}.json`)
.then(res => res.json())
.then(data => res.status(200).json(data))
})
module.exports = router
// postcode model
const divPostcode = document.querySelector('.postcode')
const formPostcode = document.querySelector('.postcode-form')
const inputPostcode = document.querySelector('.postcode-input')
const suburbList = document.querySelector('.suburb-list')
formPostcode.addEventListener('submit', handleSubmit)
async function handleSubmit(evt) {
evt.preventDefault()
let postcode = inputPostcode.value
suburbList.innerHTML = ""
// make endpoint to send a postcode then make a call to the suburb api
fetch(`/api/postcode/${postcode}`)
.then(res => res.json())
.then(data => {
for (let suburb of data) {
let name = suburb.name
let nametagLi = document.createElement('li')
nametagLi.className = 'suburb-name-li'
nametagLi.textContent = name
suburbList.appendChild(nametagLi)
}
})
}
You see, for literally every other API we used, all we would need is the second step: using fetch to get the data from an API
endpoint, turning it into readable json, and then using DOM Manipulation, like in Tic-Tac-Toe, to update the site with the fetched data.
But for Postcode API? Nope. Had to use a proxy to get the data then retrieve the data from the proxy. Now before, I start ranting further,
I suppose I should bring up some code that I was happy about (mostly because I fixed the problems mostly by myself)...
// spotlight model
const spotlightStation = document.querySelector('.randomStation')
export async function getSpotlight() {
const result = await fetch('/api/stations/random')
const data = await result.json()
let name = data.station_name
let address = data.address
spotlightStation.innerHTML = `${name}
${address}`
return data
}
// map model
// spotlight code
import { getSpotlight } from './spotlight.js';
const spotlightStation = document.querySelector('.randomStation')
const refreshLink = document.querySelector('.refresh')
refreshLink.addEventListener('click', async () => {
spotlightData = await getSpotlight()
spotlightLat = spotlightData.lat
spotlightLng = spotlightData.lng
})
// data for spotlight
let spotlightData
let spotlightLat
let spotlightLng
// initial spotlight
spotlightData = await getSpotlight()
spotlightLat = spotlightData.lat
spotlightLng = spotlightData.lng
spotlightStation.addEventListener('click', () => goToStation(map,spotlightLat,spotlightLng))
So there was this feature we had called 'Spotlight'. It would pull a random station from our database which users could then click to move the map to. The trouble was that updating the map required access to the map object we had initialized at the very beginning. The Spotlight data and the map were in different files so that was a problem. This was solved by importing the function into the map file but then another problem arose: the Spotlight coordinates would only be set once.
If we refreshed the Spotlight for a new location, clicking it would still take the user back to the first Spotlight location. This was because we were initializing the Spotlight data on the first call and we couldn't reinitialize the variables. This was solved by initializing the variables as empty variables to be filled up later.
Lastly, we had problems getting the Spotlight to actually move since moving the map required coordinates and eventListener functions couldn't take arguments. This was solved by just calling a function that called the moving map function.
The funny thing is that I didn't even work on the Spotlight initially so when I got to it, I had no idea what it was. In the end, it was frustrating but ultimately satisfying to solve, which I could say for a lot of the project too.
Rapture
Before I close off on Petrol Down Under, I'd like to give a shout out to the rest of TEAM JESUS. Initially, I was really worried about the group project because I don't usually work well in groups. However, the peer programming turned out to be extremely helpful; I never felt under too much pressure nor did I feel like I wasn't contributing enough. I could egg my teammates and they could egg me back just fine. Overall, 10/10 experience. Would become a disciple again.
And now, a little angry comment about Google Maps API for having the worst variable names I have ever seen. Like seriously, why is everything like 'xx' or 'Xh' or 'ii'? And why does it change randomly?! Our app broke once overnight because a variable changed and, as of writing this now a month later, it's been broken yet again because of a variable change. If I never have to work with Google Maps API ever again, it will be too soon.
Anyhow, what's done is done. I forgive Google Maps. Now feel free to move on to my other projects: Project 1: Tic-Tac-Toe, Projecte 2: GameShare™, or Project 4