Introducing Aria, your new generative AI assistant in Hookpad

[Ryan]

Hello all,

The past year we’ve been working hard on a project that we are very excited to finally reveal!

As we’ve all witnessed, the recent surge in artificial intelligence technology is reshaping our problem-solving approaches. This evolution opens lots of exciting opportunities for AI-assisted co-creation across various disciplines, and music composition is no exception.

This past year, we’ve partnered with Chris Donahue, a professor at Carnegie Mellon University, to adapt the open-source Anticipatory Music Transformer into a powerful AI assistant for Hookpad, affectionately named Aria. Aria is designed to analyze your project’s current state, offering musically consistent and coherent chords and melodies to inspire and expand your creative palette. Aria is also capable of generating music with no context (i.e. from an empty project), to help spark your creativity.

At Hooktheory, we are deeply committed to honoring your creative integrity and autonomy throughout the songwriting process. Our vision for Aria is not to overshadow your creativity but to serve as a catalyst for exploration and innovation. We’ve had a lot of fun playing around with Aria and we hope you will too.

Effective today, 03/27/24, Aria is available in beta to all Hookpad owners and subscribers, at no additional cost. To use it, click on Aria in the top bar, and simply make a selection of notes, chords, notes + chords, or measures, and choose from the generation options in the Aria context GUI. Please note that during the beta, we will be collecting data on how long it’s taking Aria to respond, as well as what suggestions it is offering, to ensure that the model is working properly and that users aren’t waiting too long.

Post-beta, Aria will transition to a subscription-based model separate from Hookpad. We understand the hesitancy towards additional fees, but the reality is that hosting Aria demands substantial GPU resources, and the subscription fees will directly support the computational costs, ensuring Aria remains an efficient and effective for you in the longterm.

We hope you enjoy Aria as much as we do, and please feel free to leave us feedback on your experience.

Happy songwriting!
Ryan, Chris and Dave

[neurozero]

This may be the melody writing tool I’ve been looking for. This is literally the only one that comes up with a decent melody over borrowed and altered chords, and I’ve tried them all.

It is pretty buggy though.

More “Rendering…” issues:

It has also deleted my existing music a few times, but not always. I’ll try to narrow down steps that cause it to happen.

That said, congratulations on creating the very first useful melody generator.

[Sy_Mercade]

I have the app downloaded, but Aria appeared on it. Came here looking for what its about, it does seem cool, but I pay for hookpad and have downloaded the app, can I be assured that that’s not going to go away in the future? I understand I wouldn’t be able to stay updated on it, but I thought by downloading it as an offline application I could use the version I have forever even if the site changes, is that not true? I do understand I couldn’t use it if I stopped paying, but the price isn’t going to be increasing with these new features if you don’t pay extra for the AI will it? Also don’t understand now, is it downloaded to my computer or is it always calling a server to use? Thanks

[neurozero]

Here is a workaround for the “Rendering…” bug. Zooming in and out doesn’t work to temporarily fix it like it does in the main panel, but this does:

1. Scroll way down in your document (at least a couple pages)
2. Scroll to the top as fast as you can, e.g. by dragging the scrollbar thumb with your mouse
3. From there, regardless of where your selection is, keep the document scrolled nearly to the top or it will stop working.

Scrolling up slowly doesn’t work for some reason.

---

Another bug: There is inconsistent behavior of the Aria panel when opening or creating new files when the panel is open. I planned to confirm these before posting, but was cut off so this is just from memory:

• New:
  • Aria panel is closed
• Open (From Server):
  • Aria panel left open and suggestions cleared
• Open From Disk:
  • Aria panel left open and suggestions still available

---

Another bug: Suggestions are often verbatim copies of melodies elsewhere in the composition. In one project, I was offered 10 out of 29 suggestions - over 1/3 - that were identical or octave transpositions of existing melodies. I understand trying to match existing style and all, but those should probably be filtered out, especially if they count toward the daily cap.

---

Last bug (for 24 hrs at least!): When the cap is reached, it deletes the list of suggestions already offered.*

*What specifically happened to me is this: I had a bunch of suggestions in one tab, but hit the cap in a different tab. I flipped back to the tab with all the suggestions in it and tried scrolling down the list to see if I could view the last few already generated suggestions (to get an even 30 sample size for the previous bug). Instead, when scrolling down triggered the generation a new suggestion that failed, all existing suggestions were erased and replaced with the 24 hr limit message.

[Ryan]

@neurozero thanks for the detailed bug reports, and glad to hear that Aria is sending you some helpful ideas. We’ll be working on resolving these issues shortly. Also good to hear that you were able to run up to the limit, we could consider bumping that up to allow you to play with the tool some more, the limit was mostly to ensure that everyone had access to the feature without having to wait too much, but it seems like it’s been running pretty smoothly so far in terms of response times.

[dzakyalfajr]

Aria was bugging (same rendering issue as @neurozero). Tried refreshing the page and now the button is no longer there.

[heyzilliux]

This is so cool but I am kinda bummed out about the limit, specially because half the time it just came up with what was already there. I understand a limit though but maybe for a week we could have no limit just to play with it?

[Ryan]

@dzakyalfajr the button being gone is the view that someone who is not logged in or that doesn’t own Hookpad would see right now. Make sure that you’re not in private browsing mode, and that you’re logged in with your normal username.

[Ryan]

We’re planning on doubling the limit in the short term.

[satur9]

Quite interesting.
It seems to be producing some suggestions of note, including some really off colour ones, which isn’t a bad thing. This may be a nice way to break up a creativity block - at least if it does something I DON’T want out of a song, that seems to assist in clarifying what I do want, and I can go off on my own.

Can I suggest being able to resize the suggestions in the sidebar - at the moment I can’t work out a way to do so. The previews are so small so as not to really be indicative of the suggestions. This might mean more user clarity, less user clicking through them to determine which one is desirable, therefore less unnecessary resource usage.

Is Aria to use the same dataset that the original Anticipatory Music Transformer at Stanford used? If no (or only partially), can you give specifics on the training data?
Can I confirm there are no plans to include the private projects of Hookpad/Hooktheory users into training data?
Innovations in generative music are something I’m interested in personally, but data governance certainly is too.

Thanks guys.

[Ryan]

Thanks for the feedback. I’ll share with the team the idea of resizing the Aria bar to have suggestions scale too, that’s an interesting idea. Originally the options were just numbered blocks that you’d click on, but we added the small previews to help identify the suggestions by something a little more helpful than a number. The thinking was that most users would click on the suggestion and play through it, but I can see how you might want to reject a suggestion purely on visual inspection.

The original Anticipatory Music Transformer was trained using the Lakh MIDI dataset, a large public collection of MIDI files that serves as a gold standard in music machine learning research. Aria was built by taking AMT and fine-tuning with public theorytab data, focusing on translating more general musical information into the “lead sheet”-like vocabulary that Hookpad uses, as well as pulling in some musical ideas from popular music. Aria was not trained on any private Hookpad projects, nor would we ever use your private data for anything without your permission.

Please note however, that during this beta period, we are storing information about the quality of Aria’s suggestions and response times of the server for the purpose of improving the experience, as outlined in the Aria Terms of Service and Data Agreement.

[satur9]

Awesome. Good to know.

[floatoverblow]

Not really what I’m looking for from Hookpad, to be honest. Everyone is jumping on generative AI right now so I understand wanting to capitalize on that and make Hookpad stand out more. I’m interested in where it’s going for music, Suno and some of the other generated music is pretty unreal.

But I would like things like true polyphony, note bends, dynamics control, and breaking chords into melody data to make eg. our own arpeggiation patterns before this. Things that make the transition from sketch to production easier, anything that makes the default sounds less toy-like and patch-like will make building music from it feel so much more natural and the sell so much easier to people skeptical of dropping $5 a month for a web-app (even though that’s a strength when limited to locked-down workstations heh.)

Again, much love to the Hookpad devs, I am very happy with hookpad for what it is, I get to practice songwriting 30 minutes to an hour before work and during breaks and that’s more than most people spend on their hobbies. Hookpad is already an amazing tool for composing, I recommend it to everyone that’s looking for practice away from their DAW and for a gentler introduction to music theory. Just hope this isn’t the end to core improvements to functionality.

[neurozero]

Just got a new error:

That’s all I could get for maybe 15 or 30 minutes, but now it seems to be working again.

---

And another bug: In projects with multiple sections, the first section flag is displayed in the preview, regardless of which section the selection is in.

---

And a feature request:

There should be a way to prevent certain existing melodies from influencing the generated ones. Often when I’m experimenting, I’ll have a bunch of somewhat unrelated ideas in a project. When I stumble across a cool chord progression, I’ll try generating melodies for it to get some ideas, and often just get variations of my own half baked melodies back, sometimes even ones that were written over a completely different chord progression and clash horribly with the one I’m generating for. My current workflow requires lots of browser tabs.

The section headings might be perfect for this. Aria could perhaps ignore everything outside whatever section the selection is in, or even better, it could provide a checklist of the sections, and let the user decide which ones should influence the results. That way, they could have a section just for ideas that they’re not yet ready to delete, but that they don’t want influencing the output. Or, they could generate ideas for a verse based on a chorus, but then infill the rest of the verse without referencing the chorus again.

[Marr]

Well done Ryan and also kudos to the team. However, I desperately need an AI singer as I’m tone deaf, the artists I speak to can only rap, and then not in tune

[corasundae]

This is a songwriting app. There are plenty of AI singer programs, including ACE Studio and Synthesizer V.

[Marr]

Too bad the team didn’t take that approach for their other ‘instruments’, they could’ve saved themselves a lot of bother:-)

[themarkofzer0]

Aria is a nice extension to my use of hookpad. My typical use is to experiment and come up with chord progression ideas from the database in hookpad and then midi transfer them into a DAW and create/play a derivate melody using the chord progression (backwards to others who create music starting with a melody). I had a G Lydian chord progression I had been working on already in hookpad and Aria came up with some very interesting melody/rhythmic ideas (hooks?) to go along with my progression and helped finish a piece to publish on my youtube channel (and mention hookpad too). Thanks for the new feature to experiment with.

[fossile]

asa note Synth V has a rap feature as well as sing…