Ma\u010dina has used MathTutorBench to test the LLMs for learning from OpenAI and Google, among others. This testing revealed significant differences. \u201cWe often see that there\u2019s a trade-off between the various criteria \u2013 one model might perform very well in terms of mathematics expertise, but not in terms of its pedagogical abilities. In another model, it might be the other way around. They usually fail to achieve a balance.\u201d It\u2019s also striking, he says, that most models lose track and drift off at some point when dealing with multi-stage answers.\u00a0\u00a0<\/p>\n
\u201cA better balance between expertise and teaching abilities than traditional LLMs\u201d\u00a0 <\/p>\n
In a second project with the same team, Ma\u010dina developed a proprietary LLM that aims for a better balance between pedagogics and didactics, on the one hand, and technical expertise, on the other. He trained the model by having a virtual pupil interact with a virtual teacher in multiple steps, making do without expensive training data. The model learns from the simulated interaction and with feedback from a second model, which monitors the teaching\/learning process and evaluates the virtual teacher\u2019s responses. The LLM is therefore learning on a continuous basis in a process known as reinforcement learning.<\/p>\n
\u201cThe big advantage is that we don\u2019t need huge quantities of data and can make do with far smaller language models,\u201d Ma\u010dina explains. For comparison, the latest LLMs from OpenAI or Google have hundreds of billions to trillions of parameters. To put it simply, parameters are a measure of an LLM\u2019s cognitive abilities. Ma\u010dina\u2019s model makes do with just seven billion parameters.\u00a0<\/p>\n
\u201cWith our model, we see that there\u2019s a better balance between technical expertise and teaching abilities than with traditional LLMs.\u201d It is also less prone to drifting off course, he adds \u2013 even in the case of a learning interaction with 20 steps, it doesn\u2019t lose track. During the learning process, the model can also be asked about the reasons for certain answers and decisions. \u201cThis allows teachers to track and monitor the learning process,\u201d says Ma\u010dina\u00a0\u00a0<\/p>\n
Will there soon be an AI tutor for Master\u2019s students?\u00a0 <\/p>\n
Ma\u010dina\u2019s LLM is now freely available under the name TutorRL and has already been downloaded more than a thousand times. \u201cTo date, TutorRL is one of the few LLMs that are freely accessible and optimised for learning,\u201d he says. However, he admits that the model is yet to be tested and evaluated with learners in a classroom setting. To this end, he is currently looking for partners at schools. So far, the system also only works for teaching maths at upper-secondary and early Bachelor\u2019s level. However, Ma\u010dina can certainly imagine that, in the longer term, the model will also be used in other STEM (science, technology, engineering and mathematics) subjects and will also be sufficiently powerful for use at the Master\u2019s level.\u00a0\u00a0\u00a0<\/p>\n
In his view, however, the results are not only relevant to teaching, but also of broader value for the further development of artificial intelligence. Collaborative problem-solving, like that of TutorRL, will be essential for many areas of work in the future, as human judgement will continue to be a vital component.\u00a0\u00a0\u201cWhat we really want is satisfactory collaboration between humans and the LLMs \u2013 not for the models to do the thinking for us,\u201d says Ma\u010dina.<\/p>\n","protected":false},"excerpt":{"rendered":"Ma\u010dina has used MathTutorBench to test the LLMs for learning from OpenAI and Google, among others. This testing…\n","protected":false},"author":2,"featured_media":81209,"comment_status":"","ping_status":"","sticky":false,"template":"","format":"standard","meta":{"footnotes":"","_share_on_mastodon":"0"},"categories":[6],"tags":[225,3405,8255,1202,51],"class_list":["post-81208","post","type-post","status-publish","format-standard","has-post-thumbnail","category-zurich","tag-artificial-intelligence","tag-d-gess","tag-d-infk","tag-news","tag-zurich"],"share_on_mastodon":{"url":"https:\/\/pubeurope.com\/@ch\/116729638175992848","error":""},"_links":{"self":[{"href":"https:\/\/www.europesays.com\/ch\/wp-json\/wp\/v2\/posts\/81208","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/www.europesays.com\/ch\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/www.europesays.com\/ch\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/www.europesays.com\/ch\/wp-json\/wp\/v2\/users\/2"}],"replies":[{"embeddable":true,"href":"https:\/\/www.europesays.com\/ch\/wp-json\/wp\/v2\/comments?post=81208"}],"version-history":[{"count":0,"href":"https:\/\/www.europesays.com\/ch\/wp-json\/wp\/v2\/posts\/81208\/revisions"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/www.europesays.com\/ch\/wp-json\/wp\/v2\/media\/81209"}],"wp:attachment":[{"href":"https:\/\/www.europesays.com\/ch\/wp-json\/wp\/v2\/media?parent=81208"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/www.europesays.com\/ch\/wp-json\/wp\/v2\/categories?post=81208"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/www.europesays.com\/ch\/wp-json\/wp\/v2\/tags?post=81208"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}