Grant Proposal: Act Voyager

Category: Grants

Comments (15)


I endorse this proposal. I know that when we first started to use ACT a dozen years ago, it was a real step forward for the Perl community as it relieved conference organizers of one particular burden. It was also beneficial for conference attendees; I could maintain a single account year after years.

Alas, more recently this has not been the case. We had outages at YAPC::NA in 2015 or 2016. We had outages at DCBPW in 2016. I was therefore not surprised when the TPC::NA organizers did not use ACT in Virginia this year. But that meant that the conference organizers had to re-invent certain wheels.

Consequently, I think that this project needs to start with a fact-finding phase. What exactly went wrong during those outages? That information should be translated into stories that the ACT-rewrite needs to address before it can be used in live production.


I'm sorry to say that I don't think it makes sense to continue to work on ACT (for anybody..)

It would make much more sense to me to start a completely new product, and migrate the old data into the new system.

But here are some more concrete questions regarding this grant:


  • We all agree that the current ACT codebase (and DB!) is horrible (because it's old - it was state of the art when it was written), and some of the main features (eg localisation) have much better solutions now than back in 2004(?). So I don't think it is a good idea to reuse the old database schema. If we want the data, we could export it in a new, saner schema.


  • There is already a rewrite (also done using Dancer I think) by Racke / the Dancer Conference people. Why not use this code as a basis?


  • If the implementation has a CLI and a Web API (which is a good idea), I'm not sure if Dancer is the best solution for the Web API. In any case, the proper business logic should live in fat models, and be used by CLI and API "frontends".



To sum up, I think we should let ACT die, and either write something completly new from scratch, or maybe use some other conference toolkit (there have to be a lot!), even if they might not be written in Perl.

Of course it would also be a nice project to design a new ACT-like system, but IMO this HAS to be done by a team. AFAIK there already were
several hackathons trying to improve/rewrite ACT, which all failed. So this is not an easy task, and probably something better left to a "startup" - maybe a Perl community "startup", or a commercial one (but I think the marked for conference tools is quite flooded).

As a conference organiser and attendee, I'd also love a better Perl based tool, and I'd love to hack on one. But I think it's a too big job
for one person...


The history records of performance issues and outages are certainly not great. And indeed it can cause much frustration for organisers and attendees when this is happening 'au moment suprême'.

The Dutch organising committee of TPC in Amsterdam had been challenged to move away from Act, it had a long period off outage and many more issues. But I had enough confidence in the people maintaining the service. Personally, I think, we do not give Sébastien, BooK and Laurent enough credit for the hard work the do behind the scenes.

Last year, due to a large migration project, some things did break, but were also fixed; Act is up and running ever since and but are quickly fixed when reported.

And with Act-out-of-the-Box, it will be more easy to make patches available, since current organizing team can work on things on their local machines.

Concerning the 'fact-finding-phase', I rather would leave that as something from the past. I am not sure that it will help us much in trying to figure out what went wrong there and then. I rather focus on a scalable solution that no longer needs the ancient system we need for legacy-act.

Once Act-Voyager is in a mature state, we can move on. We will have a better platform and technically only need a robust database. The old server can be shut down. - And for those that do have concerns about the websites from old days, yes, we will freeze them and serve them as static files.


This project would be very useful. ACT is a good tool but hard to get to grips with, so simplifying the process of getting started with it would be terrific.

Act-out-of-the-Box with the additional documentation on how to get started would be terrific move forward.

Given Theo's very recent experiences with ACT, it seems a good time to have him work on this.


I do not think one should feel sorry to voice his concerns.

Act-Voyager is a way forward to an entire new Act, but I would refrain using the term 'rewrite' for various reasons:

- The Database will mostly remain as they are, so the old legacy Act backend can continue working as it is
- The frontend will not be a 'rewrite' of what we have, it will be an entire new approach that relies on a REST-api ... whatever that front end will be, it most likely will not mimic the old websites, we are on warp-speed forward to modern web applications and more.

Writing a complete new product might make sense, but with this approach, we can live with both legacy and new next to each other.

Whatever one would call the objects and what other names live under the sun, my approach will deal with separation of business-object, presentation layer and storage.

My approach does mention te documentation of the REST-api, using the Open API Specification. This specifies how frontend and backend must communicate. It defines what the backend must adhere too, not how it has been implemented. How and where data has been stored is not of concern of a frontend client or CLI. But that also means that I do not have to define the exact objects being used, FAT Models, DAO's DBIC result sets. And, it also means I can use the current DataBase (which is not horribel) and if I must, I can swap the entire storage.... without breaking any APIO clients.

As far of failing hackathons, I only attended one of them. One can blame it on me, as everyone might have thought that I would arrange all. I think many will look back to that weekend in Lyon and will be able to pick things out of it that have been beneficial and learned a lot during these interesting discussions. I have not forgotten those 'take-home' experiences.

One thing I do have a lot of concern about is to make Act-Voyager accessible for everyone to play with and add valuable contributions to it:
- adding comments and patches to the OAS / Swagger file
- adding user stories related to the frontend for guests, users, organisers and admins
- help implementing
and later
- add new features

Rather letting Act die, I would say it in a bit less radical way... put Act to REST!



> the current DataBase (which is not horribel)


You're right. I guess the DB itself is ok (though I don't remember the schema in any detail), but the DB Abstraction Layer (which predates DBIx::Class and probably even Class::DBI) is horrible (at least I found it very hard to work with...)


> Rather letting Act die, put Act to REST!


:-)

But I still think that starting with a clean slate would do the project good. We do not need the old instances, we just need a dump if the current HTML (for future reference). Some of the old data might be handy (like attendee stats etc), but I don't think the valuable old data justifies keeping any compability with the old code.

> Re: Swagger / Open API Spec


Have you actually used it in some real projects? I did, and while it has some merit in the early stages of a project, I've never seen a project actually use Swagger for the whole lifecycle, or for automatically creating clients etc. But maybe other people are more focused on Swagger than me, and do manage to achive what's promised on the tin :-)

And regarding REST / API etc: Of course ACT should be just a backend API wit various clients (commandline, JS Frontend, or even a Perl/TT-based "frontend"). But if we want to actually work on this idea, a "simple" rewrite / rebrush of the old code won't suffice.

I talked with BooK (one of the original authors) at TPCiA, and he had a few very good ideas for new / other features etc. You should definitly contact him, and/or get him to feedback here.


I do not think one should feel sorry to voice his concerns.

Act-Voyager is a way forward to an entire new Act, but I would refrain using the term 'rewrite' for various reasons:

- The Database will mostly remain as they are, so the old legacy Act backend can continue working as it is
- The frontend will not be a 'rewrite' of what we have, it will be an entire new approach that relies on a REST-api ... whatever that front end will be, it most likely will not mimic the old websites, we are on warp-speed forward to modern web applications and more.

Writing a complete new product might make sense, but with this approach, we can live with both legacy and new next to each other.

Whatever one would call the objects and what other names live under the sun, my approach will deal with separation of business-object, presentation layer and storage.

My approach does mention te documentation of the REST-api, using the Open API Specification. This specifies how frontend and backend must communicate. It defines what the backend must adhere too, not how it has been implemented. How and where data has been stored is not of concern of a frontend client or CLI. But that also means that I do not have to define the exact objects being used, FAT Models, DAO's DBIC result sets. And, it also means I can use the current DataBase (which is not horribel) and if I must, I can swap the entire storage.... without breaking any APIO clients.

As far of failing hackathons, I only attended one of them. One can blame it on me, as everyone might have thought that I would arrange all. I think many will look back to that weekend in Lyon and will be able to pick things out of it that have been beneficial and learned a lot during these interesting discussions. I have not forgotten those 'take-home' experiences.

One thing I do have a lot of concern about is to make Act-Voyager accessible for everyone to play with and add valuable contributions to it:
- adding comments and patches to the OAS / Swagger file
- adding user stories related to the frontend for guests, users, organisers and admins
- help implementing
and later
- add new features

Rather letting Act die, I would say it in a bit less radical way... put Act to REST!


I endorse this project.

We had to face several server migration for Act in the last 2 years.
Each of them was painful because of the age of the code and techno, because we are not mastering the code and architecture.
From all the attempts to propose evolutions in Act or an alternative, Act Voyager is the only one still alive.
I trust Theo to provide the Perl community with a better platform to support the conferences.


> the current DataBase (which is not horribel)

There is already a perfect database abstraction layer available for the current schema. It was the first thing I did working on this project before. DBIx::Class is not only a nice ORM, it provides us also with many other features.

Interesting enough about the DBIC version is that I also introduced a new Result Object called 'Conference'. The entire ORM would implode lacking this, there should had been a relation between this object and most of the other result sets. The database schema will show all over the place a conf_id - but we had no such table for conferences.

> Rather letting Act die, put Act to REST!

New features might need new databases, but this project is not initially set up for building new features. This will certainly make it more easy to add them. And there are dozens I can think of.
Let it first get it up and running, make it better, make it faster and more importantly, make it more robust... Then add those new shiny features organisers and attendees dream off.

> Re: Swagger / Open API Spec

There are some tools out there that nicely integrate with Swagger for testing and designing, even for setting up a mocked server. But one thing that is so important, is that the collective Open API files document the API describe the interface.

Based on those Swagger files, any developer can write a new frontend and any front end can communicate with whatever backend one would be implement.

There is an initiative for a Dancer2 plugin that would build the entire Dancer app based on a Swagger file, but that is in a very early stage.

And as far as I know, it should be quite easy to create a CLI client based on a Swagger file.

But again, although these options are available, automatic generation of clients, server, running test, mocking a server... these tools are nice, but the definition of the API itself is more important.

> Act 2

BooK has indeed many interesting ideas about Act2. Some of them I have been tempted to implement but would broaden the scope beyond the the boundaries of the Perl Community - and would not justify a Perl Foundation Grant.
Act2 would be an interesting commercial project and become a competitor of MeetUp.


agreed.

- elohmrow (sorry, lost my login details)


I think Act and it's underlying data are worth quite a lot to the Perl community, and therefore worth investing in.



This project badly needs some modernization, and while this has been tried many times before (without success), I believe that the right way to actually get some movement forward is by having an invested and motivated person (like Theo) spend the necessary time to make it happen. Several people in the Perl community have tried rewriting from scratch, and all of them have failed to complete their effort, and my impression is that this is partly because of the magnitude of the problem. Act has a lot of valuable legacy and useful features, and re-implementing all of it in one go is only feasible for well-funded organizations.



As far as I'm concerned, the greatest deficit in the Perl community is the lack of available attention to spend on improving core infrastructure in the community. When someone offers to improve the situation, I'd say "go for it!" – the cost of doing nothing is much larger than the amounts we're talking about in this grant.



I'd support this grant proposal in a heartbeat.



A complete and working Act-out-of-the-Box would be a useful thing as currently the biggest problem with making any significant change is having to push the changes to the test/live servers and that means quick turn around isn't possible. And Act-out-of-the-Box would also make contributing fixes easier, as that's something I've been wanting to do.

I'm entirely unconvinced by anything else here and largely agree with domm.

Of note is that OpenAPI is currently in flux with the v2 to v3 thing and them breaking away from JSON Schema (we have discussed this in #swagger on irc.perl.org) so you're going to have legacy stuff before you even get going.


I can imagine people would doubt my motivation and passion about this project and have questions why it has not been done in the previous grant assignment.

It has been a year and a half that most of my energy has been taken up by organising The Perl Conference in Amsterdam. If anything I desired to do, it was getting Act-Voyager developed whilst organising the conference. Having had the basics setup and adding more and more features as organising the event would move on and require more of Act-Voyager. And not only would that have been for the TPCiA but it would also had applied with the Perl Conference in iAlexandria. However, organising TPCiA sucked up more energy then I anticipated.

During the conference in Amsterdam I was already planing on picking up Act-out-of-the-Box. And even after being told that the initial grant was revoked, I still did not give up on that.

Having spend many hours on that first thing and researching others aspects, writing additional modules, I am still highly motivated to cary out this important task that will benefit the Perl Community.

I think Salve is right when he says: «the right way to actually get some movement forward»


I know that Act-out-of-the-Box will be a huge benefit on itself. Over dinner, I spoke about the essence and simplicity that it would bring, together with Lance (thanks for the dinner). Therefor I have not stopped working on that and hope to have to get it out in the next week or so.

Being «entirely unconvinced by 'anything else'» and «largely agreeing with Domm» does not make sense:

- having a REST-api
- having a CLI
- having a DBIx::Class abstraction layer on a not horrible database

is not something Domm is disagreeing on, he has a different approach and want to kill current Act and start over.

And I say, the REST api gives an interface layer between any storage and the web client or CLI, allowing to let legacy Act and Act-Voyager live along side. Something very important whilst developing and events still using, and heavily relying on, the current Act.


I have personally maintained ACT web sites for 2 YAPC's, 1 TPC, and 6 or 7 PPW's. I would have welcomed this project 5 or 6 years ago. Now, I wonder if there isn't better places to spend grant money.

Recent conferences are starting to switch to commercial event management projects that provide the features that event-goers want today, mobile apps being a big part of that.

ACT did a lot of things great in the early days and that's primarily because it had many great people putting a whole lot of effort into it. But then, people ran out of steam. I can't blame them. They really put a lot of love into ACT.

One significant problem ACT has always had, and this project will not solve, is the diverse way that all of the Perl events need to use it. It takes a lot of work to transform the square peg that is a Perl conference into the round hold that ACT needs it to be. Staying tied to the same data structures will ensure this never gets fixed.

Having ACT out of the box isn't really that great of a benefit. One of the main reasons ACT has been such a treasure in the past is the nature of the centralized data. Everybody's personal data is portable from one event to the next. You get to build a history of events you've been to. Meanwhile, event organizers also get to build a history of data from one event to the next. YAPC::NA::2012 in Madison decided not to use the same ACT server as everyone else. Instead, they stood up their own server, ran the event, and then took the site down. People reacted very badly to having to create yet another act account for Madison. And not having the site any more has caused a number of different problems both in terms of lost history and lost data. Once you make act out of the box easy, you actually make ACT lose one of its biggest selling points.

I wish Theo the best. I'm sure if anybody can fix it, it's him. I know he's put a lot of work into this already, and his work is very admirable. But I'm worried that after Theo puts a lot of love into ACT, and TPF puts a lot of money into it, it's still not going to match up with event-goers expectations. And then when Theo runs out of steam, we'll still be back to using off the shelf products again.

-Dan


Sign in to add comment