Introduction: Demonstrating Machine Learning (AI) Workflows Using Hamlet
To demonstrate the versatility of Machine Learning (AI) workflows, we applied advanced processes to Shakespeare’s Hamlet. This case study highlights how these tools can be used in a creative context, pushing the boundaries of what AI can do for creative processes and sharing insights with the community.
Workflow: Transcribing and Categorizing Hamlet
AI Transcription:
We used CHAT GPT to break down the sentences of Hamlet’s “to be or not to be" speech into a database with timecode markers based on an average speech pace. This could be more accurate with a transcribed timestamped reading.
Categorization and Classification with GPT:
Using ChatGPT, we categorised sections of the conversation into thematic subclips such as Character Analysis (Hamlet, Ophelia), Major Themes (revenge, madness), Historical Context, and Modern Interpretations. Timecodes were applied to key moments in the text.
Database and EDL Creation:
The subclips were stored in a database and organised into an EDL (Editing Decision List), ready for sequencing into an educational video or presentation..
Impact: Efficiently Organising Literary Discussions
This workflow shows how the same processes used for corporate content can be applied to broadcast, journalism, academic or creative projects, organising complex discussions into categorised, manageable segments.
In addition it is possible to for example create new narratives using these texts by clustering similar themed content added to the database and build scripts from actual recorded and transcribed content.
AI-Assisted Shot Development in Runway
Introduction: Developing Shots Using AI in Runway
For our Hamlet demonstration, we next utilised Runway to develop visual sequences based on prompts generated by ChatGPT. This part of the case study focuses on how AI-driven prompts were used to influence shot design and camera decisions, once again expanding our expertise and pushing our boundaries in creative workflows.
Workflow: AI-Powered Shot Development
Shot Prompts from GPT:
We used ChatGPT to generate creative prompts for camera angles, POVs, and stylistic elements based on the themes of the discussion. These were added to our timecode segmented database EDL. For example, a wide shot might be suggested to represent the broader themes of Hamlet’s internal conflict. The database can then be queried to build staging prompts for Runway per shot.
Runway for Animation:
We fed these prompts into Runway, which generated sequences of animated content. It is possible that we could also lip sync these sequences to audio or the transcribed text - but for this demonstration we have not done this - yet.
Impact: Creative and Dynamic Visual Content
This workflow allowed for fast iteration on animated content, with the ability to adjust camera movements or perspectives in real-time based on AI-generated suggestions.
Virtual Scenic Design with Cuebric
Introduction: Virtual Scenic Design and Segmentation Using Cuebric
For our Hamlet demonstration, we explored using ChatGPT and Cuebric for generating photorealistic plates for virtual production design. This section examines how we created and segmented virtual environments for compositing into real-time video workflows.
Workflow: Photorealistic Plates and Scene Segmentation
Scene Development with ChatGPT:
ChatGPT generated detailed descriptions of the virtual environments, including settings, lighting, and mood. At First ChatGTP returned more of a sketch so we had to be more specific and ask for photorealism.
Segmentation in Cuebric:
These prompts were then able to be depth and object based segments using Cuebric. which were then segmented into layers (foreground, background) for further compositing and exported as plates and camera perspective based FBX mesh.
Integration with Smode and MAYA:
The segmented plates were exported and integrated into Smode and MAYA, where additional effects, animations, and compositing were applied, creating immersive virtual environments for use in virtual production or to keep consistent background settings for AI generated charicters.
Conclusion: AI-Powered Video Production Across Domains
AI-driven workflows provided immense value. From automating transcription and categorization to generating creative shot prompts and virtual environments, these tools enabled us to create dynamic, high-quality video content efficiently and at scale. For Creative Alchemy, the true art is in the process, and through these collaborations, we not only expanded our own expertise but contributed to the ongoing development of AI-driven workflows for the broader creative community.
Comments