# Go Web Starter Kit This is a fairly complete web development starter kit in Go. It tries to be as simple as possible without leaving out modern features. The goal is to create a setup that a normal person can learn and study core web technologies like the DOM and web servers. I would classify it as, "Old school with modern things." ## The Stack Currently I'm using the following components in a tastefully crafted stack similar to a Crepe Cake: * [goose](https://github.com/pressly/goose) -- This manages the database migrations. * [sqlx](https://github.com/jmoiron/sqlx) -- This is the database driver. * [squirrel](https://github.com/Masterminds/squirrel) -- This is the SQL query generator. * [Fiber](https://gofiber.io/) -- This is the main web API and does most everything you need. * [tailwind](https://tailwindcss.com/) -- This makes your sites perty and is easy to use, but I reject the `@apply` slander and use `@apply` to give you nice things. * [ssgod](https://lcthw.dev/go/ssgod) -- A static site generator I wrote that _only_ does static site generation and nothing else...unlike Hugo. * [ozai](https://lcthw.dev/go/ozai) -- A process manager that replaced Air because Air went crazy like Hugo. * [chromedp](https://github.com/chromedp/chromedp) -- This does your automated testing for you. I then add a few little "sweetener" APIs on top of this for simple things like, making the tests nicer or making it easier to quickly return a page from a view. ## Getting Started First, install a couple tools that'll be used everywhere, and that have huge dependencies you don't actually need in your project: ```shell go install golang.org/x/pkgsite/cmd/pkgsite@latest go install github.com/pressly/goose/v3/cmd/goose@latest ``` To start your own project do this: ```shell git clone https://lcthw.dev/go/go-web-starter-kit.git my-project cd my-project ``` Then on Linux/OSX you want to delete the `.git` with this: ```shell rm -rf .git mv LICENSE LICENSE.old # it's MIT ``` And on Windows you use the ever more clear and totally easier: ```shell rm -recurse -force .git mv LICENSE LICENSE.old # it's MIT ``` Once you have that you can make it your own: ```shell git init . go mod tidy cp config_example.json config.json make migrate_up sqlite3 db.sqlite3 ".read tools/pragmas.sql" make make dev ``` This gets the site running and ready for development. You only need _to do this once._. The next day that you want to work, cd to your project and only run `make dev`: ```shell make dev ``` ## Linux Bug? I found that on Ubuntu the `tailwindcss` command "wouldn't run." Turns out when `tailwindcss` forks it does the right thing and closes all possible open file descriptors, _but_ Ubuntu has set the hard limit to 1 billion: ```shell $ ulimit -Hn 1073741816 ``` This is a _lot_ of files to try to close, so `tailwindcss` is actually stuck doing that. You can fix this with: ```shell ulimit -n 65536 ``` You can pick any reasonable number, and then `tailwindcss` works like expected. ## Configuration There's 3 basic moving parts to the whole system: `webapp`, `ssgod`, `tailwind`, and `ozai`. You can configure the `webapp` using the `config.json` which you copied over from `config_example.json`. The options are fairly self-explanatory. You can configure [ssgod](https://lcthw.dev/go/ssgod) using the `ssgod.json` file. You'll notice that there's _two_ layouts with this setup, one for `webapp` and another for `ssgod`. You can just point `ssgod` at your `views/layouts/main.html` file if you want, but it seems people want two different layouts for their static site vs. "the app." You can configure `tailwind` using the `static/input_style.css` and editing the `Makefile` to have it run how you want. I use the `@apply` heavily because I'm a programmer and no programmer would ever tell someone they should sprinkle repetitive bullshit all over their HTML. Resist the authority. Use `@apply`. Finally, you can configure other processes to run with [Ozai](https://lcthw.dev/go/ozai) in the `.ozai.json` file. Why so many different formats? I'm still learning which is best to use and while I like TOML I think I may go with `.json` files to reduce dependencies. You tell me though. ## Ozai Autorestarts Ozai is a simpler tool that doesn't do auto restart. Instead it will create a little webserver at `http://127.0.0.1:9999` with endpoints you can hit with curl to force a restart. Look in the .ozai.json file to see what's configured. This makes it easy to simple add a `curl` call in your build process and kick everything over. Look in the `Makefile` for how I do this. This works better than the way `Air` does it for a few reasons: 1. You can run your build like normal in your own IDE or build tool and see your errors directly. When you use other autobuild tools they have the errors so you can't access them. 2. You get an immediate restart after your build succeeds, rather than waiting for an external tool to detect you made changes, run the build, and then restart your browser. 3. You avoid the "auto reload deadlock" where you run a build, the server is shutdown, but you hit refresh too soon, so now your browser waits for an error, then you have to refresh again. Way easier to just not use any of that. 4. No more silent failures hiding inside your terminal that you can't see. Just manually restart it. For more, checkout the [Ozai Project](https://lcthw.dev/go/ozai"). ## Tour of Directories The directories are organized in a way that separates "App Stuff" from "Content Stuff". * `admin` App The code for the built-in database admin tool. * `bin` App Binaries that run your app like `webapp`. * `features` App Where you put your features. * `common`App Common helper functions. * `config` App The configuration for your site, which is `config.json` loaded into Go structs. * `data` App Your database stuff. You put your SQL models in here, and then add them to the bottom `Map` to make them magically show up in the `/admin/table/` tool. * `migrations` App When you use [goose](https://github.com/pressly/goose) it puts the migrations in here. * `tests` App Your tests go in here. I use [chromedp](https://github.com/chromedp/chromedp) but the `chromedp` API is too low level (and weird) to be useful so look in `tests/tools.go` for what I use. * `tools` App Various admin automation tools go here. * `views` Content This is where you put the _App_ contents, not the static content. Sadly, this should be in `features/` but Fiber makes that too difficult. * `emails` Content This where you put templates for emails you'll send. * `pages` Content These are templates that are generated once and then served statically. * `static` Static This is static content like `.js`, images, and `.css`. The following directories are considered "build junk" and you _should not_ commit them to your git. * `public` Junk This is generated by `ssgod` and then served by your `webapp`. Don't edit anything in here, and instead edit in `pages` or `static`. * `tmp` Junk This is made during the build/testing process. ## Dev Workflow The first thing I do is run each of these commands in different terminals, either with tmux or just using my fingers to run multiple tabs in PowerShell: ```shell make dev ``` ### 1. Create a Feature The first step is to create a `feature`, which is nothing more than some files in a directory that you edit to make your `pages/` actually work. You can create a feature like this: ```shell ./bin/fgen -name myfeatue ``` This will create the files and directories you need to make a new feature: `features/myfeature/` : This contains all of the go code. `views/myfeature/` : This is where your HTML goes. `tests/myfeature/` : This is where your tests go. Next you add it to `features/init.go` to enable it: ```diff @@ -8,2 +8,3 @@ import ( "MY/webapp/features/fakepay" + "MY/webapp/features/myfeature" ) @@ -15,2 +16,3 @@ func Setup(app *fiber.App) { features_fakepay.Setup(app) + features_myfeature.Setup(app) } ``` > __NOTE__: I'm going to automate this in the future, so expect this file to be generated. After that run `make` to build everything again and you should have a new feature at `http://127.0.0.1:7001/myfeature/` with your browser. ### 2. Create Initial Views The design of this system is that you do _not_ have to edit 16 files just get an idea going. You can have an idea and create a single `.html` or `.md` file in `views/myfeature` to see it happen. This lets you work out all of the URLs, page flows, layouts, and basic design before writing any more code. This is generally a better way to work than creating the backend models first as it constrains what you create to only exactly what you need for the UI you'll implement. What I do is make a new file in `myfeature/views/`, cook up a quick 2D layout that's mostly what I want, then make it link to other `myfeature/views/` to figure out the feature. Once I have a rough UI for the feature worked out I move on to the next step. ### Using the Tailwind Starter I've included [tailwind](https://tailwindcss.com/) in this setup, since it's decent at getting things working quickly right in the HTML. The only thing I've fully rejected is the idea that "@apply is bad m'kay." No, `@apply` makes it so I don't rage at all the idiots who think 92 repeated CSS classes on every `div` is "good practice." > __NOTE__ You can view a sample page with all of these in [/examples/sample.html](/examples/sample.html) and you can look at the other files in `examples` to get an idea of what you can do. Some of these just use tailwind, others use my starter kit. In my setup there's a `static/input_style.css` that is used to generate a `static/style.css` and it comes preconfigured with many of the things you'll need to get a page up and running. In theory you can prototype a first page with just HTML and a bit of tailwind as needed. It also uses many of the stock HTML tags like `aside` and `mark` so you can keep things nice. I've added 4 additional layout tags for you to use: * `grid` -- For stuff in a grid. Add `class="grid-cols-2"` from tailwind to change its layout. * `block` -- A simple pre-configured vertical flex box. Put it around vertical stuff. * `bar` -- A simple pre-configured horizontal flex box. Pit it around horizontal stuff. * `stack` -- A simple way to stack stuff on top of other stuff. You know, like in every single 2d compositional system since the 1500s. * `shape` -- A simple 2d box to use for placeholders where you'll put images. It's a lot quicker to drop a little `shape` than to go find an image. I think it also makes it easier to focus on the design. ### 3. Create Fake Data Create a fake set of data that comes from your page prototype. For example, if I have a User Profile page then I might add this: ```html ``` This will give you something to work with while you sort out how the interactive parts of your page will work. If you don't have any then you can skip this part. ### 4. Refine the UI Interactions Using the fake data determine the user interactions for the view. Remember that this system is designed to minimize the amount of work you need on the front-end, so all you're doing here is simple things like validating a form, getting data, disabling/enabling fields, and anything that stays _on one page_. If you find yourself trying to cram multiple pages into one then you've gone too far. With my simple use-profile example I could create the function that renders the table on page load: ```javascript $boot(async() => { const tmpl = $id('table-row'); const target = $id('table-items'); for(let key in data) { $append(target, $render(tmpl, {key, value: data[key]})); } }); ``` This comes from my `jzed.js` file that's a minimalist old-school DOM helper. I also include [lit.js](https://lit.dev/) if you prefer to create little components for things. ### 5. Move the Fake Data to `api.go` You should be spending a lot of time working out the data and interactions in the nice convenience of the page you're working on, but eventually that data needs to come from the server. To continue the _User Profile_ example we can create a handler in `api.go` that returns this same fake data: ```go package features_myfeature import ( "github.com/gofiber/fiber/v2" ) type UserProfile struct { UserName string Email string } func GetApiUserProfile(c *fiber.Ctx) error { profile := UserProfile{ UserName: "Zed", Email: "zed@zed.com", } return c.JSON(profile) } func SetupApi(app *fiber.App) { app.Get("/api/myfeature/user_profile", GetApiUserProfile) } ``` This is just enough code to get a simple JSON API going that we can load from the `index.html` you've been editing. To grab the data just do this: ```javascript const data = await GetJson('/api/myfeature/user_profile'); ``` In place of your prototype data. Doing that will get the data from the `api.go` code you just wrote. ### 6. Store the Data in `db.go` Once you've sorted out how the data is going to come from your API you can start to store it in the database. The first step for that is to move your data into `db.go`: ```go package features_myfeature type UserProfile struct { UserName string Email string } func GetUserProfile(id int) (UserProfile, error) { profile := UserProfile{ UserName: "John", Email: "zed@zed.com", } return profile, nil } ``` The `UserProfile` is removed from `api.go` and placed in `db.go`. You then call your `GetUserProfile` function to get it: ```go import ( "github.com/gofiber/fiber/v2" . "MY/webapp/common" ) func GetApiUserProfile(c *fiber.Ctx) error { profile, err := GetUserProfile(1) if err != nil { return IfErrNil(err, c) } return c.JSON(profile) } ``` > __NOTE__: This is intended as a simplified way to _grow_ a feature from idea to working backend. > Eventually you would want to create a separate module for your data models, create your tests for > them, and other important design considerations. ### 7. Move Hard Stuff to `views.go` There's some things that are too hard to do in JavaScript, but are very easy to do in `views.go` usign the basic templates available. An example of this is defining the structure of a `` tag. Doing this in JavaScript is a massive pain, but doing it with a template in `views.go` is easy. Another example would be setting the `id` for some data object the page should display. It's far easier to do that right in the template. I won't get into how this is done, but look at any of the examples in `features/` and `admin/` to see me do this. ### 8. Put it In The Database Once you have your data API worked out you can then use [squirrel](https://github.com/Masterminds/squirrel), [goose](https://github.com/pressly/goose), and [sqlx](https://github.com/jmoiron/sqlx) to make it store everything in the database. First you create a migration for your data: ```shell goose sqlite3 db.sqlite3 create create_user_profile sql -dir migrations ``` You then edit the `_create_user_profile.sql` file to create the new table: ```sql -- +goose Up -- +goose StatementBegin CREATE TABLE user_profile ( username TEXT email TEXT ); -- +goose StatementEnd -- +goose Down -- +goose StatementBegin DROP TABLE user_profile; -- +goose StatementEnd ``` You use `goose` to migrate the database up: ```shell goose sqlite3 db.sqlite3 -dir migrations up ``` If you make a mistake, you can migrate "down" to undo your last migration: ```shell goose sqlite3 db.sqlite3 -dir migrations down ``` I also have convenient `Makefile` actions for this: ```shell make migrate_up make migrate_down ``` ### 9. Rewrite `db.go` to Store It The next step needs refinement, so I'll only briefly describe it now: 1. You have your data, so annotate it with `db: ""` struct tags. 2. Use the `squirrel` and `sqlx` APIs to store it. 3. Add your usual CRUD operations. I'm going to improve this so most of this can be automated, so stay tuned. For an example of doing this look in `admin/table/db.go`. ### 10. Refine, Refine, Refine Finally, nothing in this process is meant to be a lock-step one and done operation. This is simply a mechanism to help people go from abstract idea to concrete working code with all the stages necessary. Once you have everything "working" you'll need to refine it. The key to refinement is to think about how this feature works with other features: 1. Are you repeating data that some other feature has? Just use that feature's data operations then. 2. Are you repeating components that another feature has? Try making a `lit.js` component you both use. 3. Should you move the data used by many features into a single place? Look at `data/model.go`. You should also go over the API and page interactions in your feature, and when it's good move onto the next feature. The only way to learn this step is to do it a lot, but in general, less is more. If you've got 10k lines of code in a feature then rethink it. ## Note on The Process for Pros This process is mostly for people who are learning how to build a web application. Once you've done it this way for a few web apps then you can do it however works best for your idea. Some people's ideas are about the data, so starting with the UI like this may not work. If your idea is about the data, start with the `api.go` and `db.go`, and then work the tests. Generally though, I've found that people who start with the backend are mostly only starting there because that's what they know. They also tend to create glorious crystal palaces that are completely unnecessary because they aren't grounded in the reality of a UI actual people need to use. When you start with the UI can you can prototype all of the interactions, figure out the data, and aim for only what you need. Keep doing this and you'll slowly build a nice data model with less cruft for the application you're creating. ## Additional Features ### Sending Emails Coming soon... ### Receiving Payments Coming soon... ## FAQ Here's some questions I get when people see this. ### Isn't Fiber Banned?! First off, nobody has the right to "ban" a project. If you're hanging out in the Go discord and you've seen Fiber on some ban list then whoever did that is a fucking asshole. Just ignore that bullshit and use the best tech you can. Second, if you're only supposed to use the Go standard library then why [is there a whole fucking document on the official go website explaining how to use Gin to make an API?](https://go.dev/doc/tutorial/web-service-gin). It seems like whoever is claiming that you are only allowed to use the Go standard library is in disagreement with the _actual fucking Go project_. Finally, you don't have to do what you're told, especially when the person telling you what to do is some random asshole hiding in a chat room being a random asshole. Use what you want, and Fiber is in that sweet spot of being a nice API that isn't slow dogshit like [Gin](https://www.techempower.com/benchmarks/). ### Why Not Gin? Gin is a piece of crap, that's why. Look at the [Techempower Benchmarks](https://www.techempower.com/benchmarks/) to get an idea of how bad Gin is. It's at the _bottom_ of the list, performing even _worse_ that some of the slowest _Ruby_ web frameworks out there. Make a _compiled language_ like Go perform that bad takes a special kind of incompetence. ### Why Not Only the Go Standard Library? Because I like nice things, and while the Go Standard Library has everything you _need_, it frequently is missing things you _want_. This project is about adding the things you'll probably want without getting to abstract and far away from learning the core web technologies. ### Are Your Tags "Accessible"?! What about SEO?! Yes, I actually tested all of this with [NV Access](https://www.nvaccess.org/) and the only thing NV Access seems to have a problem with is the `pre` and `code` tags. That's because apparently those are deemed as "images" so they get ignored...which is about the dumbest fucking shit I've ever heard. I'm working on a way to fix that. Anyone telling you that my `grid`, `bar`, `block`, or `stack` tags impact accessibility most likely has never ran NV Access one time so they're just wrong. Additionally, it's better to have these be _ignored_ by screen readers because they're just 2D layout junk, so they're kind of irrelevant to a linear 1D screen reader. Finally, SEO isn't impacted because Google has literally said HTML quality doesn't matter for SEO ranking. Think about it, if Google went around dictating exact HTML they would get in so much Monopoly trouble. Instead they tend to focus more on content and organization, and I believe the official quote is, "Making everyone use the same HTML would make the web boring." However, take this with a grain of salt because the same Google people who've said this also said that backlinks didn't impact performance and many SEO experts say this is totally not true. Do your own testing, and if it impacts your SEO than just change them after you get your initial design up.