The 12-Month Challenge sale (Jan 1-15th) ended.

I wish you all a Happy (and hopefully less chaotic) New Year!

A big thank you to everyone who supported this little website last year! Your support is truly appreciated. I am heartened to see the continued support despite my lack of updates.

I sincerely wish I had more time for Koohii last year but it was very rocky times for me. Still is but things are getting better slowly. I'm in a safe spot financially and I don't have any significant health problems so I am grateful for that.

Resolutions!

One good thing coming out of 2020 ; the long and short of it ; is that I am planning regular development time on Kanji Koohii until next summer! If things go well maybe all of 2021. I really don't know.

I have grown confident in working from home, so I have put aside approximately 15 hours a week to progress on the website. I am excited to have some regular time again to work on Koohii!

With that said, the task ahead is pretty daunting... let's have a look.

The state of things

The site's code is kind of a mess right now. On the backend side we're still using Symfony 1.x, which hasn't been maintained since Nov 2012! For Javascript we're still using components of YUI2, a library from Yahoo that has been deprecated in 2011. There is some Webpack, VueJS and TypeScript in there, mostly as an experiment, as it was added far too late in the project.

The biggest issue right now is that I am unable to take advantage of so many good packages out there. Anything from advanced authentication, managing an API and API tokens for connected apps, job queues, a good admin panel, better user registration & authentication, etc.

To be fair I was also a bit out of the loop myself. For example I never used a CSS pre-processor to write the site's stylesheets! I caught up a lot in the last couple years of employment. I am now much more familiar with Sass, and especially Tailwind. I am also familiar with TypeScript now, though by no means an expert.

I am still behind the times with php, but mostly for lack of practice. I just read about php Traits recently. Catching up on php shouldn't be too difficult. I will make efforts to have peer reviews from developers in the community to learn & improve.

Reboot!

The site needs to migrate to a modern framework. There are many issues and features that I've kept postponing as it wasn't easy to do in the first place using an out of date framework, and as time goes by it doesn't get better. Today, I have no other choice.

I call this project Koohii Reboot, very imaginative I know. It will keep me busy for the first half of 2021.

I created a simple Gantt chart to not lose track of the bigger goal, but for now let me introduce the three main phases ahead in good ol' plain text.

Phase 1 of 3 : Packing Up

In this first phase I want to check a few items off my to do list:

• Delete Account feature: most sites these days let you delete your account at the click of a button. For a long time I was able to hande these requests via the contact form, but I have been lagging behind lately and it's admittedly not what I would expect myself from other websites/apps. I have to implement this feature to sort of "automate" this process.

• Finish Typescript API refactor: this is something that was halfway done. I'd hate to throw away the code, and I need a refresh on Javascript + Typescript anyways. Mainly this is about using javascript "Promises", to make the Typescript code less verbose.

• Issue #184 Add "Again" button in Review page : I feel like this is an important enhancement for the Review page. I've always wanted the SRS portion of the site to do better, and this will give me a little dopamine boost from releasing a feature before I start the next phase which will take months to really show its value.

Phase 2 of 3 : Laravel Migration

This is the meat of the work that will keep me busy in the coming months!

We will have a new repository, using the latest version of Laravel framework.

The main two options come down to Symfony or Laravel. I like Symfony but we're almost certainly going to use Laravel for two big reasons:

1. Laravel seems overall better suited for a solo developer. Especially a "front end" developer like me who isn't experienced in the backend and devops areas. Anything the framework provides, especially packages and "starter kits" that I don't have to write myself, will be a huge asset.

   At least on paper, Laravel will let me write code faster. I really like Symfony, but Laravel seems to have way more usable packages out of the box. API authentication and API token management (think mobile clients) with Sanctum. A complete app "starter kit" called JetStream that includes registration, email verification, user profiles (with ability to upload an avatar), etc. For a dashboard/admin panel, I will be able to take advantage of Nova, Orchid or Backpage. For local development I should be able to use Homestead, or use Laravel Sail as a starting point for configuring Docker containers. The list goes on.

   There are many debates online between Symfony and Laravel. I am aware of the architectural discussions regarding facades, and dependency injection. I understand Laravel is an opiniated framework, and I have no problems with that. Both Symfony and Laravel are great choices, but for Koohii I think Laravel will be more practical.

2. Laravel also seems to benefit from a more diverse community around the world, which I think will better match the developers among the Koohii userbase. Overall I think I will get more knowledge sharing via Laravel, as well as learning resources like Laracasts. And if someone is inclined to make a contribution, especially improving my code, I think it's more likely to happen on a Laravel codebase.

It looks like I am learning Laravel just in time as well, with Laravel 8 released in September, and the Jetstream 2.0 package just released.

Migration you say? Are you nuts Fabrice!?
Yes, I am very much aware that refactoring the entire site to a new framework is an impossible task. It will take forever, AND on top of that it simply makes no sense.

Since the site will need a visual refresh at some point, as well as a redesign of the "user journeys" (new and returning users)... if we were to simply refactor the site it would not only take forever, it would also be a massive waste of time as we would be doing much of the work twice.

It makes a whole lot more sense to have the possibility of using a modern framework and front end stack immediately for developing new features, while retaining existing code. Then over time we can gradually drop old functionality for redesigned screens that use the newer stack (Laravel, Vue, Sass, Tailwind, etc).

Not nuts! Figs!
What we are going to do is called an "in place migration". Also called the Strangler Fig pattern. Here is a good article explaining how the "hand off" could work specifically with the Laravel framework.

From the user's perspective once this new Laravel version is published, there should be no functional difference. Some of the settings pages may look different as part of the code will run from Laravel and use a different layout. This should not affect users too much as those pages will be part of the registration & user profile screens. The Study & Review sections should run just the same, and continue to be "served" by the legacy framework.

One possible change all users may notice is that I may enable the user profile photo. This is yet another example of basic functionality on most sites that I just can't write on the old codebase, it would take me forever. Laravel Jetstream includes profile photos. I always thought the site could use a little bit of community feel so that will be a first step.

(Note: there will be some immediate benefits to users, but not very obvious ones. The authentication and registration in general will be improved. There will be email verification. The session management should be better, hopefully I can solve once-and-for-all users losing the session if they sit out for too long on the Review page for example.)

From a developer standpoint there will be a huge difference. We will be able to immediately take advantage of the new Laravel features, especially for things that do not require UI such as better authentication and API for connected apps. I will be immediately able to start an Admin panel. Take advantage of the better tools like job queues. I will be able to develop new features immediately on Laravel, and use Laravel features and facades within the legacy code.

Likely the route taken here will be to write new settings and user profile pages in Laravel. The main reason is that if those pages look different it's not much of a big deal.

The main challenge in this phase is to merge the Koohii user session into the Laravel's handling of users & authentication.

I think I will also enable the Teams feature in Laravel Jetstream, and simply hide it until it becomes useful. From what I understand, it will let me add moderator roles to the site, so special users could have additional functionality displayed to them for moderating and/or improving content.

I estimate this phase will take approx. two months. It will begin by learning Laravel, and watching some laracasts! But honestly if I take twice as much time to do it, it's still a big win in 2021.

The completion of phase 2 will be a new repository on Github, with Laravel as the framework. Much of the existing code will move to a /legacy subfolder. The repository will have one final commit, which should be the new "master" branch that should mirror the functionality of the site exactly as it is now (plus the improvements to authentication & registration from Laravel Jetstream).

From this point on, I will invite experienced php & Laravel developers who want to help to follow the repository. It would be very helpful to have peer review of the code I wrote and pointing out problems, or builtin features in Laravel I didn't know about.

Phase 3 of 3 : Better Hosting & Deployment

Once we have a production ready Laravel-based repository, I will begin to look into better deployment and hosting solutions.

I have talked about this last year when the server had some hiccups but didn't have the time to look into it deeply. But to be fair it will make more sense now: with a Laravel repository, I should be able to take advantage of deployment & hosting solutions such as Laravel "Forge" and/or "Envoyer".

I have no experience whatsoever in "devops" so anything that I can use to simplify the process of setting up and monitoring a server, as well as deploy and update the website will be very beneficial. It will need to be as hands off as possible, so if it ends up being a CPanel alternative so be it. It may also be that maintaining an account on a fully managed shared server is the safer option.

We can summarize this phase in four points:

1. Find a new host: I will look into services like DigitalOcean, Amazon AWS, etc. Of particular interest here, is services that have builtin deployment & hosting for Laravel apps. I should also experiment with Laravel Forge.

2. Find a simple build & deploy strat: I will look at Laravel Envoyer. Though the simpler solution I can find the better. I'm not sure whether it's worth using a paid service for deployment alone.

3. Deploy a public test site w/ Laravel front: this step involves a deployment practice run of the actual Laravel-based repository with all current site features working (through the legacy code "hand off"). The site will be on a public testable URL shared with the community (you!).

4. Finalize production build & deployment: at this stage the solutions for deployment and hosting are set, and the production site (this one here!) will be deployed through these new tools. After that, all hell breaks loose. ... Just kidding.

This last phase of the "reboot" I estimate might take another two months. Essentially I have no choice but to try many things and see what works and what doesn't.

I think it might be a good idea by then, to keep this site here (shared hosting) running and available as a "read only" backup, as I get to grips with the newer server and figure out how much resources it needs (in fact come to think of it, I should probably keep this site here in its original non-Laravel form running as a backup, just on another domain).

I should be able to complete this "Koohii Reboot" in less than six months (by June 2021). If it takes me longer, then so be it, I will be very happy as the site will be in a much better place in terms of longevity and stability.

Long term : Redesign

The main challenge after this reboot, is that sections like the Study page and the SRS & Flascard Review pages can't be refactored piecemeal. This is the point at which I will make mockups in FIGMA and rethink the "user journeys" (new and returning users). Instead of refactoring the existing screens to Laravel, I will hit two birds with one stone and rewrite new screens entirely.

A more realistic outcome soon after the "reboot", is that someone may contribute an API, enabling applications like Kanji Ryokucha with proper authentication using the API functionality from Laravel Sanctum. We have a very crude API of sorts right now, for Kanji Ryokucha, which isn't ready for public consumption. On the Laravel codebase it will be much better, and with proper caching. Someone also requested if it was possible to make a command line client? Why not. With Laravel Sanctum the user can create "tokens" for apps so this should be much more doable, as the core functionality is provided.

Over the coming years then, as new screens and features are implemented on the Laravel side, the legacy code will be deleted. As the article mentioned earlier says, removing legacy code should provide for some nice endorphin releases :)

The big summary

Summarizing this mammoth of a blog post:

• For the first half of 2021, I will migrate the Kanji Koohii functionality to a modern php platform called "Laravel".
• During this time I'm afraid I won't be able to develop any significant new features, though the site will benefit from better registration & authentication.
• Once this "rebooted" work is done, the website is more likely to see contributions.
• One of the most realistic outcomes perhaps before the end of 2021, is that we will have a proper API with authentication via tokens -- allowing for the creation of third party apps.
• The website may run better, depending on what hosting solutions we are going to use.
• Redesigning the user journeys (new and returning) is a longer term goal that may begin late 2021. If I need a break from programming I may create some mockups in FIGMA, that I will share with the community to bounce some ideas.

Whew. That was quite the blog post. It took me forever to write. I was a bit frustrated as I wanted to get started asap with items on phase 1 -- that said writing the post got me thinking about important topics so I already started looking into the Laravel Migration.

Godspeed to you and to me!