My guess is that anything requiring nontrivial business/technical domain knowledge will be fairly safe. Also anything with a visual (or auditory) correlate, like UI work.
Yeah, the example given in the OpenAI GPT4 twitter video is someone asking it to write a python script to analyze their monthly finances and it simply just importing dataframes, importing "finances.csv", running a columnar sum for all finances and then displaying the sum and the dataframe. I'm sure it's capable of some deeper software development but it almost always makes radical assumptions and is rarely ever self sufficient (you don't need to look it over and don't need to change the architecture of the code it produced).
Why would you think this? As long as the technical domain knowledge is at least partially published, I don't see them stopping becoming better.
UI stuff just has an input problem. But it is not that hard to think that ChatGPT could place widgets once it can consume images and has a way to move a mouse.
> As long as the technical domain knowledge is at least partially published
Most internal technical and business domain logic of companies isn’t published, though. Every time I asked ChatGPT about topics I had actually worked on over the past decade or two, or that I’m currently working on, it basically drew a blank, because it’s just not the category of topics that are discussed in detail (if at all) on the internet. At best it produced some vague generalisms.
> once it can consume images and has a way to move a mouse.
That’s quite far from ChatGPTs current capabilities, which is strongly tied to processing a linear sequence of tokens. We will certainly improve in that direction as we start combining it with image-processing AIs, but that will take a while.
Check out the announcement. GPT-4 accepts mixed-mode inputs of text and images.
Mouse cursor instructions aren’t a massive leap from the current capabilities, given the rate of progress and recent developments around LLM tool use and the like.
I wonder if there will be a race to buy defunct companies for access to their now valuable junky tech-debt ridden hairball code, so they can train on it and benchmark on fixing bugs and stuff. With full source control history they could also find bug resolution diffs.
That source code isn’t worth much without the underlying domain knowledge, large parts of which only exist in the employees’ heads, more often than not. Maybe if the code is really, really well documented. ;)
Companies could in principle train an in-house AI with their corporate knowledge, and will likely be tempted to do so in the future. But that also creates a big risk, because whoever manages to get their hand on a copy of that model (a single file) will instantly have unrestrained access to that valuable knowledge. It will be interesting to see what mechanisms are found to mitigate that risk.
I think what you say goes for most jobs. Why would GPT know much detail about being a machinist or luthier?
Eventually job and role specific information will be fed into these models. I imagine corporations will have GPTs training on all internal communications, technical documentation, and code bases. Theoretically, this should result in a big increase in productivity.
>UI stuff just has an input problem. But it is not that hard to think that ChatGPT could place widgets once it can consume images and has a way to move a mouse.
I remember one of the OpenAI guys on Lex Fridman podcast talking about how one of the early things they tried and failed at was training a model that could use websites, and he alluded to maybe giving it another go once the tech had matured a bit.
I think with GPT-4 being multi-modal, it's potentially a very close to being able to do this with the right architecture wrapped around it. I can imaging an agent using LangChain and feed it a series of screenshots and maybe it feeds you back a series of co-ordinates for where the mouse should go and what action to take (i.e. click). Alternatively, updating the model itself to be able to produce those outputs directly somehow.
