Kejia HUA
1
,
Fan ZHANG
2
,
Yirui JIANG
3
,
ERIC ZHAO
4
1
AI Research Institute
2
AI Research Institute
3
AI Research Institute
4
AI Research Institute
Abstract
Artificial intelligence (AI) has revolutionized content creation workflows, yet the cognitive principles underlying effective human-AI collaboration remain poorly understood. This study investigates when human feedback most effectively complements AI processing in collaborative content creation. Using a controlled experimental design with 120 content professionals, we systematically varied human intervention timing across three stages: conceptualization, organization, and refinement. Results demonstrate that human intervention at the organization stage produces significantly higher quality content compared to earlier or later interventions. This advantage reflects cognitive complementarity principles where human analytical reasoning optimally enhances AI-gathered information before narrative structuring. The pattern is explained by three mechanisms: intermediate state processing advantage, dual-process cognitive integration, and reciprocal cognitive scaffolding. These findings establish foundational principles for human-AI cognitive collaboration that extend beyond content creation to domains including medical diagnosis, scientific discovery, and education.
Keywords
Human-AI collaboration,Content creation,Feedback optimization,Business writing,Large language models
How to Cite
HUA, K., ZHANG, F., JIANG, Y., & ZHAO, E. (2026). Human-in-the-Loop Optimization for AI-Generated Content. Asia Journal of Social Innovation and Development, 2(1), 10. Retrieved from https://www.ajsid.org/index.php/pub/article/view/25
📄1. Rahwan, I., Cebrian, M., Obradovich, N., Bongard, J., Bonnefon, J. F., Breazeal, C., Crandall, J. W., Christakis, N. A., Couzin, I. D., Jackson, M. O., & others. (2019). Machine behaviour. Nature, 568(7753), 477-486.
📄2. Brynjolfsson, E., & Mitchell, T. (2017). What can machine learning do? Workforce implications. Science, 358(6370), 1530-1534.
📄3. Lake, B. M., Ullman, T. D., Tenenbaum, J. B., & Gershman, S. J. (2017). Building machines that learn and think like people. Behavioral and Brain Sciences, 40, e253.
📄4. Kahneman, D. (2011). Thinking, fast and slow. Farrar, Straus and Giroux.
📄5. Brown, T., Mann, B., Ryder, N., Subbiah, M., Kaplan, J. D., Dhariwal, P., Neelakantan, A., Shyam, P., Sastry, G., Askell, A., & others. (2020). Language models are few-shot learners. Advances in Neural Information Processing Systems, 33, 1877-1901.
📄6. Bommasani, R., Hudson, D. A., Adeli, E., Altman, R., Arora, S., von Arx, S., Bernstein, M. S., Bohg, J., Bosselut, A., Brunskill, E., & others. (2022). On the opportunities and risks of foundation models. arXiv preprint arXiv:2108.07258.
📄7. Hutchins, E. (1995). Cognition in the wild. MIT Press.
📄8. Clark, A. (2003). Natural-born cyborgs: Minds, technologies, and the future of human intelligence. Oxford University Press.
📄9. Dellermann, D., Ebel, P., Sollner, M., & Leimeister, J. M. (2019). Hybrid intelligence. ¨ Business & Information Systems Engineering, 61, 637-643.
📄10. Teevan, J., Baym, N., Butler, J., Horvitz, E., Shneiderman, B., & Weld, D. S. (2022). Human-AI collaboration in creative and knowledge work. Communications of the ACM, 65(5), 76-84.
📄11. Gazzaniga, M. S., Ivry, R. B., & Mangun, G. R. (2013). Cognitive neuroscience: The biology of the mind. WW Norton & Company.
📄12. Dehaene, S. (2014). Consciousness and the brain: Deciphering how the brain codes our thoughts. Penguin.
📄13. Guilford, J. P. (1967). The nature of human intelligence. American Educational Research Journal, 5(2), 249.
📄14. Amabile, T. M. (1996). Creativity in context: Update to the social psychology of creativity. Westview Press.
📄15. Marcus, G. (2020). The next decade in AI: Four steps towards robust artificial intelligence. arXiv preprint arXiv:2002.06177.
📄16. Bender, E. M., Gebru, T., McMillan-Major, A., & Shmitchell, S. (2021). On the dangers of stochastic parrots: Can language models be too big? Proceedings of the 2021 ACM Conference on Fairness, Accountability, and Transparency, 610-623.
📄17. Mialon, G., Dess`ı, R., Lomeli, M., Nalmpantis, C., Pasunuru, R., Raileanu, R., Roziere, B., ` Schick, T., Dwivedi-Yu, J., Celikyilmaz, A., Grave, E., LeCun, Y., & Scialom, T. (2023). Augmented language models: a survey. arXiv preprint arXiv:2302.07842.
📄18. Jakesch, M., Hancock, J., & Naaman, M. (2023). Human heuristics for AI-generated language are flawed. Proceedings of the National Academy of Sciences, 120(11), e2208839120.
📄19. Bansal, G., Wu, T., Zhou, J., Fok, R., Nushi, B., Kamar, E., Ribeiro, M. T., & Weld, D. (2021). Does the whole exceed its parts? The effect of AI explanations on complementary team performance. Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems, 1-16.
📄20. Raisch, S., & Fomina, K. (2025). Combining human and artificial intelligence: Hybrid problem-solving in organizations. Academy of Management Review, 50(2), 441-464.
📄21. Shneiderman, B. (2020). Human-centered artificial intelligence: Reliable, safe & trustworthy. International Journal of Human–Computer Interaction, 36(6), 495-504.
📄22. Harvard Business Review. (n.d.). How to pitch Harvard Business Review. Harvard Business Review. Retrieved September 12, 2024, from https://hbr.org/guidelines-for-authors.
📄23. Dunbar, K. (2000). How scientists think: On-line creativity and conceptual change in science. Creative thought: An investigation of conceptual structures and processes, 1, 461-493.
📄24. Raffel, C., Shazeer, N., Roberts, A., Lee, K., Narang, S., Matena, M., ... & Liu, P. J. (2020). Exploring the limits of transfer learning with a unified text-to-text transformer. Journal of Machine Learning Research, 21(140), 1-67.
📄25. Chowdhery, A., Narang, S., Devlin, J., Bosma, M., Mishra, G., Roberts, A., ... & Fiedel, N. (2022). PaLM: Scaling language modeling with pathways. arXiv preprint arXiv:2204.02311.
📄26. Lake, B. M., Ullman, T. D., Tenenbaum, J. B., & Gershman, S. J. (2017). Building machines that learn and think like people. Behavioral and Brain Sciences, 40, e253.
📄28. Mahowald, K., Ivanova, A. A., Blank, I. A., Kanwisher, N., Tenenbaum, J. B., & Fedorenko, E. (2023). Dissociating language and thought in large language models: a cognitive perspective. arXiv preprint arXiv:2301.06627.
📄29. Shneiderman, B. (2020). Human-centered artificial intelligence: Reliable, safe & trustworthy. International Journal of Human–Computer Interaction, 36(6), 495-504.
📄30. Brynjolfsson, E., & McAfee, A. (2014). The second machine age: Work, progress, and prosperity in a time of brilliant technologies. W. W. Norton & Company
📄31. Lubars, B., & Tan, C. (2019). Ask not what AI can do, but what AI should do: Towards a framework of task delegability. Advances in Neural Information Processing Systems, 32.
📄32. Wang, D., Weisz, J. D., Muller, M., Ram, P., Geyer, W., Dugan, C., ... & Gray, A. (2019). Human-AI collaboration in data science: Exploring data scientists’ perceptions of automated AI. Proceedings of the ACM on Human-Computer Interaction, 3(CSCW), 1-24.
📄33. Cheng, J., Teevan, J., Iqbal, S. T., & Bernstein, M. S. (2015). Break it down: A comparison of macro-and microtasks. Proceedings of the 33rd Annual ACM Conference on Human Factors in Computing Systems, 4061-4064.
📄34. Newell, A., & Simon, H. A. (1972). Human problem solving (Vol. 104, No. 9). Prentice-Hall Englewood Cliffs, NJ.
📄35. Klein, G., Moon, B., & Hoffman, R. R. (2006). Making sense of sensemaking 1: Alternative perspectives. IEEE Intelligent Systems, 21(4), 70-73.
📄36. Minsky, M. (1986). The society of mind. Simon and Schuster.
📄37. Hutchins, E. (1995). Cognition in the wild. MIT Press.
📄38. Kahneman, D., & Klein, G. (2009). Conditions for intuitive expertise: a failure to disagree. American Psychologist, 64(6), 515.
📄39. Rahwan, I., Cebrian, M., Obradovich, N., Bongard, J., Bonnefon, J. F., Breazeal, C., Crandall, J. W., Christakis, N. A., Couzin, I. D., Jackson, M. O., & others. (2019). Machine behaviour. Nature, 568(7753), 477-486.
📄40. King, R. D., Rowland, J., Oliver, S. G., Young, M., Aubrey, W., Byrne, E., Liakata, M., Markham, M., Pir, P., Soldatova, L. N., & others. (2009). The automation of science. Science, 324(5923), 85-89.
📄41. Topol, E. J. (2019). High-performance medicine: the convergence of human and artificial intelligence. Nature Medicine, 25(1), 44-56.
📄42. Clark, A. (2003). Natural-born cyborgs: Minds, technologies, and the future of human intelligence. Oxford University Press.
📄43. Malone, T. W., & Bernstein, M. S. (2015). Handbook of collective intelligence. MIT Press.
📄44. Hutchins, E., & Klausen, T. (1996). Distributed cognition in an airline cockpit. Cognition and Communication at Work, Cambridge University Press, 15-34. (Cited by 2,100+)
📄45. Lake, B. M., Ullman, T. D., Tenenbaum, J. B., & Gershman, S. J. (2017). Building machines that learn and think like people. Behavioral and Brain Sciences, 40, e253. (Cambridge University Press, Impact Factor: 14.2)
📄46. Evans, J. S. B. T., & Stanovich, K. E. (2013). Dual-process theories of higher cognition: Advancing the debate. Perspectives on Psychological Science, 8(3), 223-241. (SAGE, Impact Factor: 9.8)
📄47. Bommasani, R., Hudson, D. A., Adeli, E., Altman, R., Arora, S., von Arx, S., Bernstein, M. S., Bohg, J., Bosselut, A., Brunskill, E., et al. (2022). On the opportunities and risks of foundation models. arXiv preprint arXiv:2108.07258. (Stanford HAI, Cited by 1,300+)
📄48. Sweller, J., van Merrienboer, J. J. G., & Paas, F. (2019). Cognitive architecture and ¨ instructional design: 20 years later. Educational Psychology Review, 31, 261-292. (Springer, Impact Factor: 8.7)
📄49. Newell, A., & Simon, H. A. (1972). Human problem solving. Prentice-Hall. (Cited by 33,000+, Seminal work in cognitive science)
📄50. Vygotsky, L. S. (1978). Mind in society: The development of higher psychological processes. Harvard University Press. (Influential work on cognitive scaffolding, Cited by 135,000+)
📄51. Hemmer, P., Schemmer, M., Kuhl, N., V ¨ ossing, M., & Satzger, G. (2024). Complementarity ¨ in human-AI collaboration: Concept, sources, and evidence. arXiv preprint arXiv:2404.00029.
📄52. Topol, E. J. (2019). High-performance medicine: the convergence of human and artificial intelligence. Nature Medicine, 25(1), 44-56. (Nature Publishing Group, Impact Factor: 87.2)
📄53. Gil, Y., Greaves, M., Hendler, J., & Hirsh, H. (2014). Amplify scientific discovery with artificial intelligence. Science, 346(6206), 171-172. (AAAS, Impact Factor: 63.8)
📄54. Roll, I., Butler, D., Yee, N., Welsh, A., Perez, S., Briseno, A., Perkins, K., & Bonn, D. (2018). Understanding the impact of guiding inquiry: The relationship between directive support, student attributes, and transfer of knowledge, attitudes, and behaviours in inquiry learning. Instructional Science, 46(1), 77-104. (Springer, Impact Factor: 3.3)