How often do you Google a quick question to only be bombarded by various, sometimes conflicting, answers from different websites? If your answer was “a lot” then you may be excited to hear the news about the new AI chatbot, ChatGPT, that is rumoured to replace Google as a search engine, but how effective is this AI?UK Radiators’ in-house heating experts decided to ask ChatGPT the UK’s most searched heating questions to evaluate how accurate and helpful the responses are. Here is what we found.
How Much Does Underfloor Heating Cost?
ChatGPT’s Answer
Evaluating ChatGPT’s Answer
Score: 8/10
The answer provided by ChatGPT was concise yet informative. It clearly understands that there are several factors to consider when it comes to underfloor heating and does a great job of breaking it down. It also gave us a rough estimate of how much everything would cost, which would be very helpful to anyone considering purchasing underfloor heating.
However, it didn’t provide any additional knowledge or explanation in its answer. Our expert highlights a few factors to consider when looking at running costs while ChatGPT did not mention this in any detail. When considering buying underfloor heating, it is vital to know how much it will cost each month before installing it, hence why we think it should have been mentioned in the response.
How Do Heat Pumps Work & Are They Worth It?
ChatGPT’s Answer
Evaluating ChatGPT’s Answer
Score: 6/10
ChatGPT provides a short and simple explanation of heat pumps, which is great for people who have zero knowledge of the topic. However, this information wouldn’t be very useful for someone looking to dive deeper into the topic.
However, we think there’s a lack of detail from ChatGPT here. In this case a heating expert will not only provide a more detailed explanation for each type of heat pump but will also give location-specific information that is relevant to those looking to buy a heat pump in the UK.
How much is the heating allowance & how do I claim it?
ChatGPT’s Answer
Evaluating ChatGPT’s Answer
Score: 7.5/10
The reason this was not a ten-out-of-ten answer from ChatGPT was that it did not provide as much information and detail as our expert, and simply referenced the government website instead. Having said that, it did provide a very concise and accurate answer to the question.
How Can I Heat My House For Free?
ChatGPT’s Answer
Evaluating ChatGPT’s Answer
Score: 8.5/10
This answer from ChatGPT shows that it understands the dynamic between money and heating and understands that it is very hard to save money on heating without spending money. It has also offered a few really good tips to heat your house, some of which our in-house expert included in his list.
Our expert would have offered a couple of free methods to heat your home more effectively such as moving furniture and cleaning your radiators to ensure that the answer suits all different budgets.
How To Heat Your Home Cheaply?
ChatGPT’s Answer
Evaluating ChatGPT’s Answer
Score: 10/10
ChatGPT nailed this question! This gets a 10/10 from us because the AI bot provided a wide range of suitable and actionable tips to help people heat their homes cheaply
Should I Leave the Heating On All Night In Winter?
ChatGPT’s Answer
Evaluating ChatGPT’s Answer
Score: 8 /10
ChatGPT presented a great summary of the point of view held by the majority of people and explained the main reasons behind it. However, it did miss out on a very common problem faced by people that turn their heating off at night – freezing pipes.
In addition, our expert would offer a more comprehensive view on the issue by offering some external links to resources to help the reader make their own mind up for their situation.
Why Is My Radiator Not Heating Up?
ChatGPT’s Answer
Evaluating ChatGPT’s Answer
Score: 4/10
This is where years of experience in the heating industry become evident. While ChatGPT’s answer offered a few helpful solutions, it did not break down the solutions to fit different use cases.
Our expert would recommend breaking down your answer by the different types of heating systems and whether the issue is evident in one or multiple radiators. He would also provide external links to other articles that offer a lot more information to the person struggling with the issue.
What Is The Best Time To Turn On The Heating?
ChatGPT’s Answer
Evaluating ChatGPT’s Answer
Score: 3/10
In this case, ChatGPT did not recognise the different types of heating systems and the different ways they work. However, it did provide some insightful information on how you generally want to think about turning on the heating.
What Is Central Heating?
ChatGPT’s Answer
Evaluating ChatGPT’s Answer
Score: 8/10
ChatGPT answered the question perfectly, but it did not provide as much supplemental information as our in-house expert would have liked. With heating definitions, it’s important to understand how other people may use the term as some may be using the old definition of central heating.
Why Is My Heating Working But I Have No Hot Water?
ChatGPT’s Answer
Evaluating ChatGPT’s Answer
Score: 5/10
In this instance, ChatGPT’s answer is a little too broad to help someone trying to fix an issue. This is why the step-by-step guidance and the depth of a “How To” article are great when trying to find a fault with your heating.
Is ChatGPT ReadyTo Replace Heating Experts?
ChatGPT scored 68/120 on our test which is slightly over 57% so we asked an expert in AI to evaluate how ChatGPT did compare to our expert to understand more about AI and its limitations.
Scott Jones, web expert and CEO of the UX agency Illustrate Digital, had this to say:
“Whether AI can reach a point where it can fully replace all our experts is the great debate around ChatGPT right now, but I would argue not… Whilst it’s easily the most powerful chatbot ever created, it’ll only ever be as good as the data it’s fed. Experts combine knowledge, relevance and context to provide expert opinions. Whilst ChatGPT will become increasingly accurate with facts and handling common conversations such as support requests, it’s unlikely to ever reach a point that it replaces experts entirely. After all, someone still needs to feed AI the information it needs to reach greater levels of accuracy.”
In conclusion, while ChatGPT can provide accurate and informative responses to a wide range of heating-related questions, it is important to note that a heating expert may be better equipped to handle more complex or specific queries. Moreover, a heating expert can also understand your individual needs a lot better than ChatGPT which is why we always recommend finding the opinion of an expert you trust before you make any decision.
