Post image

Starting my first real project! : webdev

I started at the end of 2019 my work on Devolio, a social community for programmers! I found other online tech communities sometimes too clickbait or dull (Case in point of a post I just found:Top 13 useful JavaScript array tips and tricks you should know) or they were not focused on actual discussion (talking to people about tech is super interesting!).

So I want to build this platform for programmers to share and engage with other fellow coders. I am building this with a Rails backend and the only JS framework I am using is Jquery (outdated stack but it works). All this will be hosted on Heroku for now but I will switch to more scalable alternatives later.

I am really excited about the different features I am working on to make my project fun for me to build and for other coders to use:

  • Design customization (think MySpace) secured with this great lib (CSS can be dangerous!)

  • public and open API with documentation (opens up avenues for data analysis and tons of interesting projects)

  • open source collaboration possibilities (creating a friendly environment to build a hopefully awesome project)

  • markdown everywhere with the Redcarpet parser

Progress & Pics

School takes up a decent chunk of time but I am still happy with my progress and the overall design and structure of the project.

Post image
Post image
Post image
  • Avatars (see below). I spent quite a bit of time trying to get the perfect user avatar setup. A banal user without a github username or a custom pic, is assigned an identicon from this beautiful software (this is what you see at the start of the gif). If they give their github username, the github avatar will be displayed instead, and of course they can also add their own pics.

Future Goals

Here are the features and important parts of the project logic I need to work on:

  • A well-designed comment model

  • enhancing the project design (users can share and design pages for projects they are working on)

  • Adding AJAX for follows and likes

  • setting up AWS S3 for image uploads in posts

It is really exciting to be working and deploying the first project I am really serious about! I can’t wait. Of course, all this shiny functionality won’t matter if people don’t find the project useful (paradoxical considering community-based projects are only useful if they have users). I want to see if there is real interest in building this tool and would love to get the help of other devs for this tool!

If you have any questions, please comment down below and I’d really appreciate to hear your thoughts!

Source link

Post image

I’m a front-end engineer who loves building side-projects. M…

Hi r/webdev, I’m a front-end engineer who loves building side-projects. My latest is an AI Art Generator. In this article I talk about the technology choices I made while building it, why I made them, and how they helped me launch the app a lot faster than I otherwise would have been able to. Note: I originally posted this on Medium. I’ve stripped all mentions of the actual app to comply with this sub’s self-promotion rules.

Post image

First, a brief timeline

October 14, 2019 — Looking back at my commit history, this is the day I switched focus from validating the idea of selling AI-generated artworks, to actually building the app.

October 28 — 2 weeks later I sent a Slack message to some friends showing them my progress, a completely un-styled, zero polish “app” (web page) that allowed them to upload an image, upload a style, queue a style-transfer job and view the result.

October 30 — I sent another Slack message saying “It looks a lot better now” (I’d added styles and a bit of polish).

November 13 — I posted it to Reddit for the first time on r/SideProject and r/deepdream. Launched.


A lot of functionality is required for an app like this:

  • GPUs in the cloud to queue and run jobs on

  • An API to create jobs on the GPUs

  • A way for the client to be alerted of finished jobs and display them (E.g. websockets or polling)

  • A database of style transfer jobs

  • Authentication and user accounts so you can see your own creations

  • Email and/or native notifications to alert the user that their job is finished (jobs run for 5+ minutes so the user has usually moved on)

  • And of course all the usual things like UI, a way to deploy, etc

How did I achieve all this in under a month? It’s not that I’m a crazy-fast coder — I don’t even know Python, the language that the neural style transfer algorithm is built in — I put it down to a few guiding principles that led to some smart choices (and a few flukes).

Guiding Principles

  • No premature optimisation

  • Choose the technologies that will be fastest to work with

  • Build once for as many platforms as possible

  • Play to my own strengths

  • Absolute MVP (Minimum Viable Product) — do the bare minimum to get each feature ready for launch as soon as possible

The reasoning behind the first four principles can be summarised by the last one. The last principle — Absolute MVP — is derived from the lean startup principle of getting feedback as early as possible. It’s important to get feedback ASAP so you can learn whether you’re on the right track, you don’t waste time building the wrong features (features nobody wants), and you can start measuring your impact. I’ve also found it important for side-projects in particular, because they are so often abandoned before being released, but long after an MVP launch could have been done.

Now that the stage has been set, let’s dive into what these “smart technology choices” were.

Challenge #1 — Queueing and running jobs on cloud GPUs

I’m primarily a front-end engineer, so this is the challenge that worried me the most, and so it’s the one that I tackled first. The direction that a more experienced devops engineer would likely have taken is to set up a server (or multiple) with a GPU on an Amazon EC2 or Google Compute Engine instance and write an API and queueing system for it. I could foresee a few problems with this approach:

  • Being a front-end engineer, it would take me a long time to do all this

  • I could still only run one job at a time (unless I set up auto-scaling and load balancing, which I know even less about)

  • I don’t know enough devops to be confident in maintaining it

What I wanted instead was to have this all abstracted away for me — I wanted something like AWS Lambda (i.e. serverless functions) but with GPUs. Neither Google nor AWS provide such a service (at least at the time of writing), but with a bit of Googling I did find some options. I settled on a platform called Algorithmia. Here’s a quote from their home page:

Data scientists never have to worry about infrastructure again

Perfect! Algorithmia abstracts away the infrastructure, queueing, autoscaling, devops and API layer, leaving me to simply port the algorithm to the platform and be done! (I haven’t touched on it here, but I was simply using an open-source style-transfer implementation in tensorflow). Not really knowing Python, it still took me a while, but I estimate that I saved weeks or even months by offloading the hard parts to Algorithmia.

Challenge #2 — The UI

This is me. This is my jam. The UI was an easy choice, I just had to play to my strengths, so going with React was a no-brainer. I used Create-React-App initially because it’s the fastest way to get off the ground.

However, I also decided — against my guiding principles — to use TypeScript for the first time. The reason I made this choice was simply that I’d been noticing TypeScript show up in more and more job descriptions, blog posts and JS libraries, and realised I needed to learn it some time — why not right now? Adding TypeScript definitely slowed me down at times, and even at the time of launch — a month later — it was still slowing me down. Now though, a few months later, I’m glad I made this choice — not for speed and MVP reasons but purely for personal development. I now feel a bit less safe when working with plain JavaScript.

Challenge #3 — A database of style-transfer jobs

I’m much better with databases than with devops, but as a front-end engineer, they’re still not really my specialty. Similar to my search for a cloud GPU solution, I knew I needed an option that abstracts away the hard parts (setup, hosting, devops, etc). I also thought that the data was fairly well suited to NoSQL (jobs could just live under users). I’d used DynamoDB before, but even that had its issues (like an overly verbose API). I’d heard a lot about Firebase but never actually used it, so I watched a few videos. I was surprised to learn that not only was Firebase a good database option, it also had services like simple authentication, cloud functions (much like AWS Lambda), static site hosting, file storage, analytics and more. As it says on the Firebase website, firebase is:

A comprehensive app development platform

There were also plenty of React libraries and integration examples, which made the choice easy. I decided to go with Firebase for the database (Firestore more specifically), and also make use of the other services where necessary. It was super easy to setup — all through a GUI — and I had a database running in no time.

Challenge #4 — Alerting the client when a job is complete

This also sounded like a fairly difficult problem. A couple of traditional options that might have come to mind were:

I didn’t have to think about this one too much, because I realised — after choosing Firestore for the database — that the problem was solved. Firestore is a realtime database that keeps a websocket open to the database server and pushes updates straight into your app. All I had to do was write to Firestore from my Algorithmia function when the job was finished, and the rest was handled automagically. What a win! This one was a bit of a fluke, but now that I’ve realised it’s power I’ll definitely keep this little trick in my repertoire.

Challenge #5 — Authentication, Notifications and Deployment

These also came as a bit of a fluke through my discovery of Firebase. Firebase makes authentication easy (especially with the readily available React libraries), and also has static site hosting (perfect for a Create-React-App build) and a notifications API. Without Firebase, rolling my own authentication would have taken at least a week using something like Passport.js, or a bit less with Auth0. With Firebase it took less than a day.

Native notifications would have taken me even longer — in fact I wouldn’t have even thought about including native notifications in the MVP release if it hadn’t been for Firebase. It took longer than a day to get notifications working — they’re a bit of a complex beast — but still dramatically less time than rolling my own solution.

For email notifications I created a Firebase function that listens to database updates — something Firebase functions can do out-of-the-box. If the update corresponds to a job being completed, I just use the SendGrid API to email the user.

Creating an email template is always a pain, but I found the BEE Free HTML email creator and used it to export a template and convert it into a SendGrid Transactional Email Template (the BEE Free template creator is miles better than SendGrid’s).

Finally, Firebase static site hosting made deployment a breeze. I could deploy from the command line via the Firebase CLI using a command as simple as

npm run build && firebase deploy

Which of course I turned into an even simpler script

npm run deploy

A few things I learned

The speed and success of this project really reinforced my belief in the guiding principles I followed. By doing each thing in the fastest, easiest way I was able to build and release a complex project in under a month. By releasing so soon I was able to get plenty of user feedback and adjust my roadmap accordingly. I’ve even made a few sales!

Another thing I learned is that Firebase is awesome. I’ll definitely be using it for future side-projects (though I hope that this one is successful enough to remain my only side-project for a while).

Things I’ve changed or added since launching

Of course, doing everything the easiest/fastest way means you might need to replace a few pieces down the track. That’s expected, and it’s fine. It is important to consider how hard a piece might be to replace later — and the likelihood that it will become necessary — while making your decisions.

One big thing I’ve changed since launching is swapping the front-end from Create React App to Next.js, and hosting to Zeit Now. I knew that Create React App is not well suited to server-side rendering for SEO, but I’d been thinking I could just build a static home page for search engines. I later realised that server-side rendering was going to be important for getting link previews when sharing to Facebook and other apps that use Open Graph tags. I honestly hadn’t considered the Open Graph aspect of SEO before choosing CRA, and Next.js would have probably been a better choice from the start. Oh well, live and learn!

Source link