Homem ou Máquina? Um estudo exploratório do desempenho do Chat GPT 3.5 no Exame de Suficiência do CFC<p>Man or machine? An exploratory study of GPT 3.5 chat performance in the CFC sufficiency exam
Keywords:
Inteligência Artificial. Contabilidade. Tecnologia. ChatGPT.Abstract
Chat GPT é um chatbot (programa de computador que tenta simular um ser humano na conversação com as pessoas) de inteligência artificial desenvolvido pela OpenAI que é projetado para fornecer interações conversacionais semelhantes às humanas. Ele é construído em cima dos modelos GPT-3.5 e GPT-4 da OpenAI e funciona reunindo dados da internet escritos por pessoas e usando previsões de computação para responder a perguntas e consultas inseridas pelos usuários. Mas, afinal, seria a máquina capaz de superar o ser humano na resolução de testes ou exames de qualificação? Para responder essa questão, o presente artigo tem como objetivo explorar o nível de acerto do ChatGPT nos exames de Suficiência (1º Exame de Suficiência de 2022) e de Qualificação Técnica do CFC (edição de 2023). Os resultados mostram que o chatbot obteve aproveitamento superior a 50% em todos os testes aplicados, índice suficiente para aprovação de um candidato. Esta pesquisa complementa artigos anteriores e demonstra a aplicação da tecnologia de IA e Processamento de Linguagem Natural na área contábil e fornece evidências relevantes para o aperfeiçoamento do ensino em contabilidade, considerando a máquina aliada do ser humano no desenvolvimento de novas habilidades.
References
ALSHATER, M. Exploring the Role of Artificial Intelligence in Enhancing Academic Performance: A Case Study of ChatGPT. [s.l: s.n.]. Disponível em: <https://ssrn.com/abstract=4312358>.
ALSHURAFAT, C. H. The usefulness and challenges of chatbots for accounting professionals: application on. [s.l: s.n.]. Disponível em: <https://ssrn.com/abstract=4345921>.
AYDIN, Ö.; KARAARSLAN, E. Is ChatGPT Leading Generative AI? What is Beyond Expectations? [s.l.] APA, 2023. Disponível em: <https://www.researchgate.net/publication/367525655>.
BAIDOO-ANU, D.; ANSAH, L. O. Education in the Era of Generative Artificial Intelligence (AI): Understanding the Potential Benefits of ChatGPT in Promoting Teaching and Learning. SSRN, 2023.
BOMMARITO, J. et al. GPT as Knowledge Worker: A Zero-Shot Evaluation of (AI)CPA Capabilities. 11 jan. 2023.
BOMMARITO, M. J.; KATZ, D. M. GPT Takes the Bar Exam. SSRN Electronic Journal, 29 dez. 2022.
CONSELHO FEDERAL DE CONTABILIDADE. Regulamenta o Exame de Suficiência como requisito para obtenção de Registro Profissional em Conselho Regional de Contabilidade (CRC). Brasília, 15 maio 2015. Disponível em: <https://www1.cfc.org.br/sisweb/SRE/docs/Res_1486.pdf>. Acesso em: 26 maio. 2023
CONSELHO FEDERAL DE CONTABILIDADE. NBC PA 13 (R3) – Exame de Qualificação Técnica Para Auditor. , 2020. Disponível em: <https://www1.cfc.org.br/sisweb/SRE/docs/NBCPA13(R3).pdf>. Acesso em: 30 abr. 2023
CONSELHO FEDERAL DE CONTABILIDADE. Edital EQT Auditoria e Perícia No 1/2023 - Exame de Qualificação Técnica Para Registro no Cadastro Nacional de Auditores Independentes (CNAI) e no Cadastro Nacional de Peritos Contábeis (CNPC) do Conselho Federal de Contabilidade. , 2023. Disponível em:
COPELAND, B. J. Artificial intelligence (AI) | Definition, Examples, Types, Applications, Companies, & Facts | Britannica, 2023. (Nota técnica).
DOWLING, M.; LUCEY, B. ChatGPT for (Finance) research: The Bananarama Conjecture. Finance Research Letters, v. 53, 1 maio 2023.
FARSEEV, A. Is Bigger Better? Why The ChatGPT Vs. GPT-3 Vs. GPT-4 “Battle” Is Just A Family Chat. Disponível em: <https://www.forbes.com/sites/forbestechcouncil/2023/02/17/is-bigger-better-why-the-chatgpt-vs-gpt-3-vs-gpt-4-battle-is-just-a-family-chat/?sh=63b43e275b65>. Acesso em: 26 maio. 2023.
FRANKENFIELD, J.; SCOTT, G. Artificial Intelligence: What It Is and How It Is Used. Disponível em: <https://www.investopedia.com/terms/a/artificial-intelligence-ai.asp>. Acesso em: 26 maio. 2023.
GARATTONI, B. O futuro da inteligência artificial – e o que vem depois do ChatGPT | Super. Disponível em: <https://super.abril.com.br/tecnologia/o-futuro-da-inteligencia-artificial-e-o-que-vem-depois-do-chatgpt>. Acesso em: 26 maio. 2023.
GEORGE, B. ST.; S. GILLIS, A. What is the Turing Test? | Definition from TechTarget. Disponível em: <https://www.techtarget.com/searchenterpriseai/definition/Turing-test>. Acesso em: 26 maio. 2023.
GIATTINO, C. et al. Artificial Intelligence. Our World in Data, 2022.
GUO, X. Research on the Transition from Financial Accounting to Management Accounting under the Background of Artificial Intelligence. Journal of Physics: Conference Series. Anais...Institute of Physics Publishing, 28 nov. 2019.
HEAVEN, W. D. ChatGPT is everywhere. Here’s where it came from | MIT Technology Review. Disponível em: <https://www.technologyreview.com/2023/02/08/1068068/chatgpt-is-everywhere-heres-where-it-came-from/>. Acesso em: 26 maio. 2023.
HUTANU, A. How ChatGPT works and AI, ML & NLP Fundamentals | Pentalog. Disponível em: <https://www.pentalog.com/blog/tech-trends/chatgpt-fundamentals/>. Acesso em: 27 maio. 2023.
LOPES, A. ChatGPT: conheça a inteligência artificial capaz de escrever artigos, roteiros e músicas | Exame. Disponível em: <https://exame.com/tecnologia/chatgpt-conheca-a-inteligencia-artificial-capaz-de-escrever-artigos-roteiros-e-musicas/>. Acesso em: 26 maio. 2023.
MADHAVI, S.; KUMAR, V. QUALITATIVE REVIEW OF LITERATURE FOR ARTIFICIAL INTELLIGENCE IN TODAY’S BUSINESS SCENARIO. Perspectives on Business Management & Economics, v. IV, p. 66–76, 2021.
MASLEJ, N. et al. Artificial Intelligence Index Report 2023 Introduction to the AI Index Report 2023. [s.l: s.n.]. Disponível em: <https://aiindex.stanford.edu/wp-content/uploads/2023/04/HAI_AI-Index-Report_2023.pdf>. Acesso em: 15 maio. 2023.
MELO, L. R. D. DE; PAGNAN, C. L. Prática de Texto: Leitura e redação. 3a Edição ed. São Paulo: A!, 2008.
MERTENS, F. THE USE OF ARTIFICIAL INTELLIGENCE IN CORPORATE DECISION-MAKING AT BOARD LEVEL: A PRELIMINARY LEGAL ANALYSIS WP 2023-01 Working Paper Series The use of artificial intelligence in corporate decision-making at board level: A preliminary legal analysis. [s.l: s.n.]. Disponível em: <https://ssrn.com/abstract=4339413>. Acesso em: 26 maio. 2023.
NUNES, D. et al. Evaluating GPT-3.5 and GPT-4 Models on Brazilian University Admission Exams. 29 mar. 2023.
OLIVEIRA JUNIOR, J. C. R. DE. ChatGPT: Desempenho do chatbot no exame de suficiência do Conselho Federal de Contabilidade. Revista Finanças Fecap, [s.d.].
PAVLIK, J. V. Collaborating With ChatGPT: Considering the Implications of Generative Artificial Intelligence for Journalism and Media Education. https://doi.org/10.1177/10776958221149577, v. 78, n. 1, p. 84–93, 7 jan. 2023.
PERLMAN, A. THE IMPLICATIONS OF CHATGPT FOR LEGAL SERVICES AND SOCIETY. 2022.
SCHROER, A. Artificial Intelligence (AI): What Is AI and How Does It Work? | Built In. Disponível em: <https://builtin.com/artificial-intelligence>. Acesso em: 26 maio. 2023.
SCIALOM, T.; CHAKRABARTY, T.; MURESAN, S. Fine-tuned Language Models are Continual Learners. Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing, EMNLP 2022, p. 6107–6122, 24 maio 2022.
THORP, H. H. ChatGPT is fun, but not an author. ScienceAmerican Association for the Advancement of Science, , 27 jan. 2023.
VIANNA, H. M. Flutuações de julgamentos em provas de redação. Cadernos de Pesquisa, v. 19, p. 5–9, 1976.
ZAREMBA, A.; DEMIR, E. ChatGPT: Unlocking the Future of NLP in Finance* ChatGPT, OpenAI. ‡. [s.l: s.n.]. Disponível em: <https://ssrn.com/abstract=4323643>.
Published
How to Cite
Issue
Section
License
Definitions. The following terms shall be used as Agreement for this submission.
Notice: As part of the process for submission, the authors are obliged to verify the submission conformity in relation to all items listed in this requirement. The submissions which are not in accordance with the rules will be sending back to the authors.
1. It is stated that are the exclusive owners and holders of copyrights and content submitted to be published and in this sense agree with free of rights which get the first edition to the publisher and publication to the publishing house of Universidade Estadual do Centro-Oeste – Publishing UNICENTRO, by means of Capital Científico Journal with the work licensed at the same time under Creative Commons Attribution License, allowing the content sharing with the recognition of the authorship and initial publication of this Journal.
In this way, the author (authors) authorizes (they authorize) that Publishing UNICENTRO, through the Capital Científico Journal (Electronic-On Line or Printed) the publication of this content by printed and electronic, also for communication to the public, including the public exhibition, reproduction by any process, distribution, and others systems of display or exposure to public, by any ways, medias and supports.
2. It is declared that the responsibility by the originality of the work, and it is declared that its content was not published partially or in parts by a second journal, as well is not being considered for publication in another journal or publisher in any form of media. The authors are responsible by the information accuracy as well the data contained in the submitted content.
3. It is declared that in the text process of writing it was observed with accuracy the rules referred to citations and references to research sources used, according to the Brazilian law 9.610/ 1988 (law of Author’s rights) as well as it was observed the dispositions contained in the Committee and Ethic reference to the object researched.
4. It is declared that the authors rights was acquired reserved to the indirect owners when in the in the work there are remissions, translations, collections, inserted photos and images, which the property is not owned by the authors, as well as, authorizations with the direct owners or titular from registered, names of organizations or companies that are mentioned.
5. It is declared and that authors are obliged to indemnify UNICENTRO or its publishers, in the case of suffering any material or moral injury in consequence of the law or extra law acts promoted by third ones in reason of its publication departing from the published work.
