I figured I’d take some of these AI app builders for a run to see how capable they are. To date I’ve mainly just been taking advantage of the AI capabilities that are progressively being added to VS Code and Visual Studio though tooling such as Copilot but I’m hoping that these AI app builders, at least for basic applications, can cut hours, days maybe months off the typical app development cycle. In this post I’ll cover what I see as the pros and cons of the app building experience.
In this first post we’re going to pick on co.dev that claims to be able to convert “text to app in minutes”.
First up, as you can imagine, there’s a mandatory sign-up/login flow – AI isn’t cheap and they want to ensure you’re not a bot or some other AI service. As I’ll be testing out their services, I don’t mind this requirement but it does make me wonder whether organisations will be open to their devs having to sign into this online service, effectively giving over access to some of their enterprise data. Data security is definitely going to be an interesting space and it’ll be a challenge for organisations to find the balance between locking down their data versus providing access to enable the generation of data driven applications by these AI tools.
I figured I’d try out building a relatively simple application that accepts a single user input, the name of a business, and calls a web api to retrieve the Australian Business Number (publically available here). Already I know there are a couple of wrinkles here, such as the web api requiring an API key and there are two different formats, SOAP and JSON, and a couple of different endpoint methods to select from. However, the web api is by no means complex, and a typical developer, using tooling built into VS (eg Add Web Service), could get an API call going in say 10-15mins.
The overall experience of starting a new project was quite good as it stepped me through a number of prompts, including setting up Supabase. Only minor quirk was that after opening a different tab to setup Supabase I had to convince co.dev to refresh in order to see my new Supabase project).
I was prompted as to what I wanted to create, so I entered “A cross-platform application that can be used to look up the ABN for an organisation”. After a few minutes a skeleton application was created. I liked the fact that it had wired up authentication, which is oftern a challenging aspect, and I was able to create an account and login. I didn’t like that despite saying I wanted a cross-platform application, it just built me a web application. I wasn’t in any illusion that I was going to get an actual cross platform application but it would have been nice for it to create something that didn’t look like a generic web portal.
I also liked that in the Chat pane it was describing what was going on, giving some insight into how the AI was thinking about/changing the application.
The current Dashboard page was blank, so I entered a prompt to build out the functionality: “On the dashboard page add an input box where the user can type the name of the business they want to search for and a submit button. When the user clicks the submit button, use the input to call the search api at https://abr.business.gov.au/Documentation/WebServiceMethods”. Again, the AI went off an processed for a few minutes before it came back and updated the dashboard page.
Despite again feeling that this was more of a web portal than an actual application, I realised that the AI was indeed adding value. I’d created the basics on this application in under 5 minutes, so who was I to complain about the way it looked.
Ok, so the real test was when I entered the name of a business and clicked the Search button.
FAIL
It raised an error – I figured this was likely to happen, so I asked AI to investigate the error (you can click on the error in the UI and add it as context to the chat, which was nice). The AI went off, churned for a bit, made some changes (switched from JSON to SOAP services) and came back claiming it had fixed the issue. I entered a name and clicked Search.
FAIL
I repeatedly tried to convince the AI to make changes to fix the issue calling the web api. This included one prompt where I told it to continue working until it could successfully call the web api.
FAIL,
FAIL,
FAIL,
FAIL,
At this point my experiment was over, and given I wasn’t paying for the premium offering, I didn’t see a way to take a look at the code to see what the issue was. The way I see it is that there are definitely some time savings to be had but as soon as AI has to go do something of even moderate complexity, it gets stuck.
Human Developers: 1 AI Code Builders: 0
Let’s try out another AI builder and see how we get on.