Today, 16-year-old Japanese singer Hatsune Miku gave another triumphant concert in Tokyo. We would hardly talk about it if the girl was real. But it is completely computer synthesized, including appearance, voice and dancing. Even her very figure during concerts is drawn with lasers in the air.
The Japanese electronic engineers and producers managed to unleash a new star in just a few weeks. And now the world show business is wondering: maybe this is the future? And live performers are no longer needed?
NTV correspondent Sergey Malozyomov I also tried to understand.
She emerges from under the stage, and hundreds of fans burst into screams of delight. The world's first star-hologram, in which everything is synthetic, and not just some parts. Hatsune Miku!
The army of fans of the virtual singer is growing rapidly. At first it was only the Japanese, and now, judging by the blogs and sales of her music on the Internet, there are also Americans, Australians, Europeans.
The technology that allows you to synthesize singing when you put text into a computer has existed for a long time. But in this project, firstly, it was improved, and, secondly, artists, composers and several real singers united to create the image, who gave the timbres of their voices.
Saki Fujita, singer: “It doesn’t matter to me that she, the virtual one, is loved and known more than the real me. Most importantly, it sounds great and people like it!”
YouTube is already full of Hatsune Miku clips, and fan love is spurred on by the fact that the Vocaloid program that synthesizes her voice has a free simplified version, and in fact Internet users can sing whatever they want with her voice. There is even a Finnish polka performed by a new star.
Hiroyuki Ito, Virtual Singer Project Manager: "Miku will open up new opportunities for composers, artists and musicians around the world, because computer synthesis will allow them to implement their ideas without the problems associated with a producer, money and finding a real artist."
Music critic Boris Barabanov watches the recording of the Japanese show with curiosity, but without delight. He says that the idea sounds attractive, but only a real performer can become a real idol for a long time. Interest quickly disappears.
Boris Barabanov, music columnist for the Kommersant publishing house: “There are Garillaz, there are these Japanese characters. You can also recall some of our Masyanya or Glucose, which at first was also a purely cartoon character. It's just one type of attraction, it's a form of pop culture, but it won't replace face-to-face interaction."
The first Russian virtual singer Glukoza recalls that for the same reasons she quickly turned into a real Natalia Ionova.
Natalya Ionova (Glucose), singer: “We drew one clip, the second. The viewer began to guess, everyone, of course, wanted to see who sings, how she really looks. The revival of the project and the release of me specifically on the stage was inevitable.
However, technology is evolving. The synthesized voice is getting more real, the 3D dances too. And how all this is beneficial for show business bosses! Such a star will never demand to raise her salary, there will be no whims, children, husbands, problems with alcohol. Although for PR, probably, everyone will come up with this too.
Reading the article will take: 3 min."The hologram was so alive that..."
Have you watched Andrew Niccol's 2002 fantasy Simone, about a virtual simulation of the greatest movie actress in Hollywood history? I watched this picture, I remember being interested and surprised by the possibilities of the film industry of the distant future ... And what would you think - in search of some interesting video in RuNet, I unexpectedly found concert videos of the Japanese hologram singer Miku Hatsune, and her song The world is mine turned out to be very interesting! Miku is the image and voice of the Vocaloid virtual simulator, one of the most popular among the program's fans.
Concert show Hatsune Miky
The development of Vocaloid software was started a couple of years before the release of the science fiction film "Simona" - by 2000, a group of teachers and students from the University of Barcelona had created a mathematical model of signal processing. The university project was funded by Yamaha Corporation, which used its results to create the Yamaha Vocaloid program, officially launched in January 2003.
What is this program? It contains the vocals of several actor's voices capable of performing a song according to the text and melody entered by the user. It should be noted that when selecting voices for the Japanese version of the program, called the Character Vocal Series, the company Crypton Future Media, which created it, experienced certain difficulties - singers and singers completely refused to participate in this, fearing that the software would create competition for them on stage. They were reassured by the promise to use only the general characteristics of the voice, not creating its complete imitation - however, the voices of the first project were, to put it mildly, mechanical.
Vocaloid 2 program
The first series of Vocaloids was created in 2004-2006, it included five voice characters - due to their primitive sounding by today's standards, all of them have long been forgotten. A completely different story with the second series of Vocaloid 2, developed from 2007 to 2011 - 22 holographic characters, including Hatsune Miku, who has experienced an upgrade during this time due to her wild popularity among users. By the way, many Miku fans believe that this holographic singer is the first and only vocaloid product. The latest series of Vocaloids on the updated engine, Vocaloid 3, currently consists of nine voice model characters, the number of which can increase - work on the series is ongoing and will continue.
Performers of the Vocaloid 2 program
Let's get back to the best product of the Vocaloid project - Hatsune Miku. The voice of this charming hologram singer was given by Saki Fujita, a Japanese seiyuu actress (a special direction in the preparation of the cast exclusively for the cartoon anime genre). Miku's vocaloid program is mostly intended for the J-pop song genre, however, it allows the performance of songs in other genres. The software character-vocaloid Hatsune Miku owes her special popularity to users of the Japanese resource Nico Nico Douga, who regularly create and publish videos with her participation and performance on the portal. Miku is involved in manga (so I know what it is), in anime, in computer games - she is a full-fledged virtual star. However, this is not the limit of popularity for the vocaloid hologram!
Is it possible for a Vocaloid program to give live concerts, to live music, and gather a serious number of viewers? You think this is nonsense - that's not it, it's all true! Solo concerts by Hatsune Miku took place several times, and in different parts of the Earth - the first appearance in the concert hall on August 31, 2009, full-fledged concerts on March 9, 2010 in Tokyo (Japan), September 18 and October 11, 2010 in San Francisco (USA), October 8, 2010 at the Anime Festival in New York (USA), March 9, 2011 in Tokyo, among other Vocaloids on January 11 and February 12, 2011. Do you think that's all? Figushki - July 2, 2011 Miku Hatsune participated in the anime concert at the venue in Los Angeles (USA), August 16-17, 2011 in Sapporo (Japan), November 11, 2011 in Singapore (Malaysia), March 8-9, 2012 in Tokyo (Japan). Cool, isn't it?
The charm of Miku Hatsune - a girl aged 16, with a height of 158 cm and a weight of 42 kg - does not pass you, the reader of the article. You will be able to personally see the charming hologram at the concert. She (the hologram singer) is unrealistically charming!
I won't lie if I choose between "singing thongs" and Vocaloids - I choose Vocaloids!
Post Views: 17 376
The question of who this is can be answered in different ways.
In three words, this is a Japanese virtual singer.
In short, this is a character that embodies the voice of the program "Vocaloid", synthesizing singing, developed by Yamaha.(Actually, the term “ Vocaloids“, which already appeared a decent amount).
If you want to know everything - sit back, the conversation will be long!
Hatsune Miku - background
Do you know that such a seemingly purebred Japanese woman like Miku has Spanish roots? The story begins in 2000, when at the University of Barcelona, with the funding of the corporation Yamaha the development of a program for the synthesis of singing began Vocaloid. The idea caught on, including in Russia, where the First Vocaloid Music Album was released and the First Live Vocaloid Concert took place.
At that time, many companies started developing software based on this technology. Fortunately for us, among them was the Japanese “Crypton Future Media”, an English source from the city of Sapporo. This company has been working with sound since 1995, developing applications for synthesizers, creating libraries of sound effects and background music for computer games and commercials, and collaborating with such giants as Sega, Sony, Nintendo, Apple, Dell, Microsoft, Roland and Yamaha. She carried out orders from the Ministry of Defense of Japan, as well as the Ministry of Education, Culture, Sports, Science and Technology.
In 2004 and 2006 Crypton created voicebanks MEIKO And KAITO accordingly, providing boxes with programs with colorful illustrations of virtual singers. Since then, the combination of the voice bank and its inherent character has been referred to as vocaloid.
For all the progressiveness of the singing synthesis method, sales of the software did not go well. But 2007 came, Yamaha updated the program engine Vocaloid» to version 2 and President of Crypton, Itoh Hiroyuki decided to go on the offensive.
Hatsune Miku - birth
Hiroyuki-san I knew perfectly well that in order to successfully promote any product, at least in Japan, you need to create a bright image that matches the product offered, so he focused his efforts on creating a new virtual star. At birth there were no small things. Firstly, she was destined to be born in the guise of a girl, since male voice banks were sold much worse than female ones. Age, 16 years old, was also not chosen by chance - such a singer can have a repertoire both on "adult" topics, and completely childish. Height was determined - 152 cm and weight - 42 kg, good indicators for Japan.
Hatsune Miku's very first illustration
Surname 初音
could be translated as " first sound" or " the first song of a bird", with the name it was even more difficult, Miku 未来
, meant "future", but if you read the syllables separately ( and, a little later, the spelling of the name changed to ミク
, similar in sound, but different in meaning, essentially like "Beatles" with its changed letter in the name), we got the number 39 (by the way, thanks to this we got Miku Day or Day 39, which is celebrated on March 9, as in Japan it is customary to indicate the month first, and then the day). And that's not all, Mi Ku is the so-called kun (actually Japanese) reading of the numbers 39, but in Japanese the on (based on Chinese) reading is also used and 39 will already be San Kyu. Reminds me of the English "Thank you", doesn't it?
And that's actually all! Crypton marketers did not give any more information, rightly believing that the fans will come up with the rest for themselves and Mika will be different for everyone, unique and, of course, the best!
The next step was to contact a well-known Tokyo illustrator Keiu Garo(Kei Garou Japanese source). Here Ito Hiroyuki had to slightly outline the character of the future star for the sketch - "sweet, but with a slightly harsh character." Kei drew two versions of Hatsune Miku, with one and two ponytails, but, as he says in, the second version was accepted ( drawing on the right
), to which Kay was unspeakably happy. By the way, the costume and color scheme of the Vocaloid was created not only by order of Crypton, but also by Yamaha, so they practically mean something.
It is worth adding that the voice of Miku was given by a little-known seiyuu (voice actress) at that time. Saki Fujita English source which at the time was 22 years old. Before working with Crypton, she managed to sing a couple of openings and endings, as well as voice minor characters in a dozen anime.
Shortly before the release of the program, Eiji Hirasawa wrote a demo song Star Shards“, which was in a series of thousands of hits of the future star.
On August 31, 2007, all the ingredients came together, the program went on sale, and Hatsune Miku was born!
First steps
Just before the start of the sales of the program, Crypton created a special website, where an illustration of Miku appeared, it was described who M was. iku hatsune and, most importantly, where users of the program could post their creations and communicate freely. By the way, the site piapro.jp still exists and new works are constantly appearing on it.
The green-haired virtual star liked the Japanese, her new songs flooded not only the Krypton site, but also the popular Japanese video hosting NicoNicoDouga from where splashed out on Youtube.
Soon, the prediction of marketers came true, Miku, completely independent of the creators, had the most famous accessory, which was not in sight when she appeared - “negi” or bow-batun, a close Asian relative of the leek. With this seemingly purely Japanese plant, Miku also acquired Russian-Finnish roots. It all started in 2006, when a flash video spread around the Internet, in which, to the song “ Eva's polka"(known to us as "Yap tsup tsop") of the Finnish folk group "Loituma" the heroine of the anime "Bleach" Orihime Inoue twirls a bow-batun in her hands. Mute the sound when opening the link! Almost immediately after the appearance of Miku, on September 6, the catchy melody was covered by the newly-minted star and posted on Nico and Youtube. Together with the melody, Miku also had leek, which somehow imperceptibly, over time, replaced the original batun.
Well, where are the Russian roots from? The melody itself appeared in Eastern Finland at the end of the 18th century, when Sweden ceded the rights to Finland to Russia and, quite often, at that time in some Finnish cities there were more Russian soldiers than local residents. The melody almost completely repeats the Russian folk dance "Smolensk Gusachok" popular among soldiers from Western Russia. The dance took root in the new soil, became popular, and in 1937 Eino Kettunen wrote a Finnish text for the polka. The joke is that the chorus “Yap tsup tsop ...” was not in the original song, it was added by the Loituma group itself and it consists of meaningless words (such as tili-tili, trali-wali). According to the members of the group, he was born during a tour in Moscow in the 90s at a rehearsal in the House of the Red Army. Unsuspecting representatives of Krypton were quick to point out that Finnish Polka, like nothing else, shows the capabilities of their software for creating entertainment content.
Illustration for the song MELT
Meanwhile, less than 2 weeks after the launch of the program, the sales revenue exceeded 57 million yen, far exceeding the revenue of any other software in Japan. And hundreds and hundreds of new songs with Miku's vocals appeared on Nico Nico. On the wave of popularity of the new program, new talented musicians appeared and gained popularity.
In particular, on December 7, 2007, the musician's song appeared on Nico Ryo"Melt" (Meruto), which instantly gained hundreds of thousands of views. But here there was a slight embarrassment. Ryo, who did not expect such popularity, used an illustration in the design of the video ( picture on the left ) artist 119 ( Hikeshi) without the permission of the latter. I had to apologize. Unexpectedly, Hikeshi became interested in the work of Ryo and, at the end of December, they united in a group Supercell, which included a total of 11 musicians and illustrators. Already in February, the song “Koi wa Sensou” (Love is War) was posted on Nico, in May “World is Mine” (Worudo izu Main), in June “Black Rock Shooter” (Burakku Rokku Shutu), (by the way, the song was written by the composer under the impression of illustration ( the one below ), and the anime of the same name was released 2 years later, already based on the song), and in August 2008 at Comiket 74, the musicians presented their first album called "Supercell", which reached the 4th place of the national hit parade.
The very first Rock Black Shooter
In December of the same year, the group posted the last song on Nico Nico Douga - "Hajimete no Koi ga Owaru Toki" (When the first love leaves), after which Sony Music seized the musicians with a stranglehold. Last song with vocals Ryo introduced at the end of August 2012, it was "Odds & Ends" (#14 on the charts), after which he finally focused on working with live vocalists ...
Almost immediately after the release of the program, on September 21, 2007, the song “Packaged” appears on Nico, from the group Livetune(as part of kz and, who left after a year and a half Kajuki P). Already at Comiket 73 in December 2007, the band presented the album "Re:package", which reached number 5 in the national chart. The musicians soon joined another giant of the music industry, Victor Entertainment, and already from under his wing released the remix album "Re: Mikus" in 2009, and the single "Yellow" in 2010. In 2012, having jumped over to the Toy's Factory studio already alone, kz presents the album "Tell Your World EP" (4th place), a year later "Reial" and in the spring of 2014 "Decorator EP". Despite working and with live vocalists, it doesn't look like kz is going to leave Mika, despite the fact that his latest album, at the time of writing, only ranked 19th.
There is nothing surprising in the fact that the popularity of the albums of the "masters" of wok music is declining, new composers appear, such as, for example, a super productive interview or an interview with his secrets of super realistic vocals..
The number of composers working with Hatsune Miku is uncountable, and Miku herself is the most productive performer in the World, every week there is at least one new album with her vocals and countless singles. The popularity of the virtual star has captured all corners of the planet, information fan sites have appeared, such as mikufan.com in English and www. America.
Concert activity
Prior to the arrival of Hatsune Miku, this part would certainly have raised eyebrows. A computer program and suddenly concerts ... Nevertheless, by 2009 the popularity of a virtual star became such that it became clear that it was time to go to the people ... The Crypton team accepted a new challenge and on August 22, 2009, as part of the festival Animelo Summer Live accompanied by live musicians took place, where she performed 2 songs. And half a year later, on March 9, 2010, a full-size solo concert known to all fans took place in the Zepp Tokyo hall!
After this landmark event, Miku traveled to all major cities in Japan, was on tour in Asia and the United States, gave 2 - 3 full-length concerts with live accompaniment per year (this is not counting concerts based on MMD technology, without musicians) and, finally, in September 2015, she performed at the main concert venue in Japan, c. The Beatles were the first Western group to perform in this hall, Hatsune Miku was the first virtual performer. It is simply impossible not to draw parallels! Moreover, the virtual singer has already been mistaken for her in the world of popular musicians - she recently performed as the opening act for Lady Gaga.
In 2012, Miku became interested in the classics. At the end of the year, she performed with the Japan Philharmonic Orchestra with download, a little later she took part in, (the link is not working yet) which was staged in the theater less than a year later in the center of Paris. There were also classical music projects with the participation of Vocaloids, but they did not receive a serious continuation.
MikuMikuDance or just MMD
MMD model TDA Miku
MMD model TDA Miku
On the wave of Miku's popularity in February 2008 Yu Higuchi introduced a 3D animation program that allows you to create a dance of a three-dimensional model Hatsune Miku and make a video, including with sound. The program is blatantly free, there are options in Japanese and English, easy to learn.
Miku's first default model - Animasa Miku was quite popular for a long time and based on it many variants of Miku were released and not only. Successful models are DT Miku- extracted from the SEGA game and, unexpectedly, (picture on the left
) - extremely popular not only for its own merits, but also for its ease of editing, the variants of which, in the form of not only Mika, but even the guys, clearly exceeded more than one thousand.
But, undoubtedly, the best model has become (the link does not work yet) from Mamama, which is not called otherwise than kawailoid (from the Japanese "kawaii" - cute)!
The program was constantly evolving, it became possible to capture movements using the Kinect sensor, the ability to put Miku in Reality, along with this, the number of models (not necessarily Miku and not necessarily Vocaloids), scenes where the model could perform, objects that she could use increased. Twice a year, almost since 2008, Nico Nico hosts video contests for md cup.
However, in May 2011, the developer of the program announced that he was leaving his brainchild. It lasted a little more than 2 years, in the summer of 2013 Higuchi-san returned with the 8th version of the program. (By the way, the latest version of the program at the time of this writing is 9.26/9.26 (64-bit) / December 2014)
And in February 2013, an anime English source was already broadcast on Japanese television, completely made with the help of MMD programs. However, we have deviated a little from the topic of our conversation.
Hatsune Miku and the Winter Festival
First illustration of Miku in winter style
First sketch of Sakura Miku
Since 2010, Miku has been actively involved in (link is not working yet) in the city of Sapporo. Now the festival is unthinkable without an abundance of Miku's ice figures, winter concerts, exhibitions and sales, as well as a tram painted with Miku's portraits. Thanks to the Winter Festival, we got acquainted with the image Snow Miku, whose hair and eyes are a deeper blue. Lately, on the official design Snow Miku competitions are held among illustrators, the theme of design changes every year.
Now I was almost ready to say that Snow Miku inspired the illustrators to create a symbol of spring - Sakura Miku, but it turned out that the first spring Miku in cherry tones ( picture on the right
) appeared much earlier - in December 2008. But the first winter version of Miku ( picture on the left
) appeared only in January 2010, apparently under the influence of the Winter Festival. The picture is not even called "Snow Miku", but "Ice Miku". There is also a summer version - Strawberry Miku with green hair and autumn, with milk tea-colored hair.
Vocals by Hatsune Miku
Initially, the virtual star appeared with one voice bank (the bank is characterized by some properties of the voice that cannot be adjusted in the program). But on April 30, 2010, an addition to the program was released called Hatsune Miku Append, consisting of six!!! additional voice banks, which greatly expanded the capabilities of the program. Now the singer's voice could have the following shades: Soft (gentle voice), Sweet (clear, childish voice), Dark (powerful, emotional voice), Vivid (bright vocals), Solid (loud, clear tone) and Light (innocent, angelic tone ).
In 2011, Crypton announced English version of Hatsune Miku. My God! He better not do it. Fujita-san, the seiyuu who gave Miku's voice, obviously had no talent for foreign languages. The work was very hard, we even had to write out specialists from America. Nevertheless, 2012 passed, demos appeared that did not stand up to any criticism, but at the end of the year, an English-language Miku appeared, accompanied by such an illustration that I can’t even imagine it in this article. The new voice bank was received rather coolly. Only in September 2013, a software update to the 3rd version of Vocaloid was released, which includes an English-language voice bank of acceptable quality.
At the time of this writing, in the fall of 2015, Hatsune Miku's program has almost been updated to the 4th version, at least the demo records promise that the new version will be very successful!
Cover of the first game in the series
Smelling the smell of money, SEGA Corporation did not stand aside, having developed a rhythm game by the summer of 2009 with the participation of Hatsune Miku and other Vocaloids. The essence of the gameplay is to press certain buttons corresponding to certain characters that appear on the screen against the background of a particular clip.
Commercial success turned the game into a whole series ...
The game was originally released on the PlayStation Portable, followed a year later by Hatsune Miku: Project DIVA Dreamy Theater for PS3 and Project DIVA 2nd. Variants of games (including those for street machines) are still being developed. New songs, costumes, items appear, graphics improve, online services appear, but the essence of the gameplay does not change.
In 2012, a chibi version appeared - the forms of Miku and her friends, Project Mirai for the Nintendo 3DS platform, a year later a continuation of this version of the game was released.
In the same 2012, -Project DIVA-F was released under PS VITA, in 2016 the X-version is being prepared for release.
SEGA has taken the iOS platform by storm with two mobile versions of Miku Flick.
The Android system is less fortunate, the Hatsune Miku Live Stage Producer version has only 25 songs and no updates are planned for it ...
The game, despite its apparent simplicity, proved to be extremely popular in Japan. By 2014, almost 5 million copies had been sold. Recently, the market has seriously expanded due to the United States, where adapted (with subtitles) games are supplied.
Miku in all seasons of the year
prospects
With all the variety of vocaloids lately, Miku confidently, by a large margin, is ahead of his virtual colleagues, seriously (at least in Japan) competing with live performers. It cannot be recognized that the effect of novelty has disappeared and interest in technology Vocaloid and the virtual star itself is gradually decreasing, nevertheless Hatsune Miku has firmly occupied her niche in musical culture.
The further fate of our pet is highly dependent on the development of technology. This mainly concerns holographic imaging. The latest work in this area will allow not only to bring the image of a virtual star into Reality, but will also make it possible to feel its touch.
How long to wait for a breakthrough and what the appearance of many green-haired singers in our world will lead to, we have yet to find out. Not too long left...
Thank you Natakhe for help and edits!
Thanks for the clarifications to Old Ghost, Dina B, Electric_A….
If you find inaccuracies or a need for an addition, write in the comments. Be sure to leave a name if you're commenting anonymously!
Users who LIKED this post.
In August 2007. Japan has been flooded with new songs and illustrations from the newly-minted virtual star.
A huge boost to the popularization of Miku was given by the appearance in February 2008 of the blatantly free 3-D animation program MikuMikuDance. Now the Internet was filling up with great speed with new video clips with Miku, and, soon, with other Vocaloids.
It was necessary to solve the question of how to bring a virtual star into reality.
Partially, this was implemented on August 22, 2009, but the most popular Vocaloid appeared on the silver screen, and the musicians remained below.
But on March 9, 2010, at a famous concert in Tokyo Miku no Hi Kanshasai 39's Giving Da y, Hatsune Miku was brought on stage after all. Let's see how they did it.
Vocaloid concerts. Visualization
The task was not easy, to place a completely virtual singer among very real musicians. In principle, some developments have already been - a projection on a transparent screen. But live musicians shamelessly shone through the virtual singer, which completely killed the whole illusion of reality.
The following scheme was applied, the projector was left, but several interesting things were done with the screen ....
They put on the stage transparent screen, so that no bright or illuminated objects fall into the field of view of viewers looking at this screen. Placed right behind him translucent spherical screen, onto which the images of the virtual singer were projected.
While the lighting was on in the hall, there was ordinary glass on the stage. But as soon as the light went out, a kind of dark window was formed in front of the audience into the Virtual World in which the Goddess appeared! (Well, at least the fans who happened to be at this concert thought so. I'm sure!)
Now, thanks to thoughtful lighting, real objects did not shine through the projections, and the sphericity of the screen created the effect of pseudo-volume. So there were never any holograms on the stage - there was an ordinary flat projection, embellished with some optical effects.
It should be added that at the first of a series of concerts there was no transparent screen or projector at all. They just used a big plasma panel that they put between the musicians.
It looked something like this:
Then the panel was either broken, or flooded with water, in short, for one reason or another, they switched to technology with projectors.
If we compare these technologies from the side of the audience in the hall, then projection is definitely better for them, since they fully enjoy the effects of pseudo-volume. (For the sake of fairness, it must be said that this effect is far from being distinguishable in the entire hall). But the audience, enjoying the concert on the screen, do not see these subtleties. But the rays from the spotlights of the projector are immediately evident. Let's compare Miku Hatsune on plasma (left screen) and in projection view (right)
One way or another, at the time of writing, the main way to visualize virtual singers is precisely the projection.
In some cases, directors bypass the narrow points of this technology (almost all concerts Magical Mirai), and, sometimes, they sit in a puddle wherever possible (failed).
But the description of modern technologies will be incomplete without mentioning the method augmented reality.
The essence of the method is that the computer, processing the real video, "writes" a virtual performer into it. Actually, the method itself is far from new and has been used hundreds of times to create computer effects in movies. What's new about it is that now the speed of the computer is enough to do this in real time, on the fly.
A little to the right of Eeyore's leg, you can see a mark on the floor - a reference point for a computer that will build the missing image .. And a little behind the dancers - white lines, boundaries that Vocaloid will not cross. As you can see, the effect is amazingly realistic!
There are a couple of pitfalls, the movements in rehearsal must be adjusted in such a way that the dancer and vocaloid do not come into contact. Of course, there will be no reaction of annihilation and a universal explosion, but a virtual singer passing through a live artist will look ridiculous ...
But the most important and offensive thing in this method is that the audience DOES NOT SEE THE PERFORMER ON THE STAGE! All the beautiful image is built only on the monitor screen. Therefore, this method can (and should) be used only for recording clips or concerts in the quasi-Live style (like a live concert), when, despite the presence of a stage and spectators (usually few), everything is sharpened for studio recording.
By the way, starting from 2016, you can often see both a projection (for the audience in the hall) and a computer model (for the viewers) at concerts. In particular, it was exactly that. Unfortunately, in these cases, TV viewers see a forked performer (clearly visible in the screenshots of the Chinese concert), although cameramen try to minimize this effect.
[I decided not to upload the video article, but to use screenshots for illustration, so if you want to watch the original video, go to Youtube using the links below the screenshots]
Vocaloids at concerts. Visualization Perspectives
As a matter of fact, since 2010, when we were shown the so-called “hologram” - a projection, nothing fundamentally new has happened.
Anti-aliasing has improved, the frame rate has increased, the screen has greatly expanded, in some places even two screens have appeared, but the principle remains the same - a two-dimensional projection.
The problem was the lack of technologies that provide the output of a holographic image in free space and the dominance of pseudo-holograms. The famous holograms in cylinders, in fact, are not holograms either. You are simply shown a two-dimensional picture from the side you look at it from.
But not everything is so bad. Salvation comes from the same Japan. Moreover, the problem has been solved since 2006, and in 2011 a working technology for creating an image in free space without a display appeared!
The saviors were National Institute of Advanced Industrial Science and Technology— AIST, university Keio And Burton Inc.. (Maybe someone knows their Kawasaki motorcycles). The essence of the method is that laser pulses in the spectrum invisible to the eye, focusing at a certain point in space, transfer air nitrogen atoms to an active state and they emit light.
On the screen on the left, you can see the wing of a butterfly (the butterfly itself is not drawn immediately), which flew at a height of 2 - 3 meters. You can watch a video of a flying butterfly with comments https://youtu.be/GNoOiXkXmYQ (by the way, we kind of cited it).
Of course, the video is not very impressive, but given that it has been 2 years since its publication and these are only the first steps, it is safe to say that the future lies with visualization using holograms in free space. And this future is not too long to wait!
Choreography at a live concert of Vocaloids
How are the dances that Hatsune Miku and other Vocaloids perform during their performances created?
It is clear that a professional choreographer is working, and then?
I am sure that many immediately remembered the program MikuMikuDance and, of course, they were wrong.
The problem is that in order to create a dance manually in this program, you will need a good knowledge of anatomy (“bones” in this program have no physiological restrictions and can be twisted as you like), excellent spatial thinking and very, very much time.
Has long been MikuMikuDance friendly with motion capture sensor Kinnect, but, anyone who has used it will confirm that the capture is very inaccurate, the subtle movements of the fingers are not captured and it will take a little less to edit the motion (dance file for the 3d model) than to create it from scratch.
Therefore, starting from the very first concert, the movements of the model were not programmed, but captured from a live professional dancer using an advanced sensor system.
Consider the process on the example of the complex PERCEPTION NEURON.
The dancer wears a special costume with 32 interconnected sensors. Sensors record the position and movements of the head, neck, shoulders, chest, waist, both hands, fingers of both hands, feet, thumbs of both feet.
The system can send data to a computer via a USB port, via Wi-Fi, or write data directly to a USB flash drive.
The data is processed by the program AXIS Neuron for their correction, correction of possible errors and conversion to most of the most popular formats for managing and developing 3-d models, including for MikuMikuDance.
On the example of the following video, which was created in honor, you can see how the data from the dancer enters the computer, and after the necessary improvements appeared in the final video.
The sensors look like this:
The sensors on the hands and fingers are especially complex:
But all these systems allow you to record movements in a file and then, having processed it on a computer, use it in a concert or video.
The concert is a live action, Miku addresses the audience according to a given program and, as a rule, everything goes more or less normally. But at the ill-fated concert in Kansai, when Miku suddenly lost his voice, the organizers would be very useful feedback
But more recently, a clip got into the network, which went completely unnoticed. Rather, Hatsune Miku simply acted as a DJ.
In fact, it was a test of the system Li-VR, in which the projection of Miku was controlled by a living person in real time. During the show, there was a glitch in the sound and Miku's lively reaction was very welcome!
Thus, in terms of choreography, a breakthrough is planned. Perhaps, very soon, Miku or other Vocaloids (more precisely, those who control the hologram-projection) will communicate with the audience live and adequately respond to unforeseen moments in the hall.
Unfortunately, it all depends on the producers of the concerts and their interest.
P.S. I mentioned that MMD is not used in serious concerts. But individual concerts are held purely under the MMD engine and without the accompaniment of live musicians. A few years ago there was a whole series of concerts Hatsune Appearans.
However, the popularity of these concerts was very low and there is no information about new ones at the moment ...
Illustration above: Dina B
If there are additions or corrections, please leave them in the comments!
Users who LIKED this post.
Hatsune Miku is a Japanese virtual singer created by Crypton Future Media on August 31, 2007. To synthesize her voice, the technology of sampling the voice of a live singer is used, namely, the Vocaloid program from Yamaha. Japanese seiyuu Saki Fujita served as the voice donor. It was first released on the Vocaloid2 engine, later added Append (Sweet, Dark, Soft, Light, Vivid, Solid), was also re-released on the Vocaloid3 engine. Discs with Miku's songs won first positions in the Japanese charts. Also, thanks to the technology of laser 3D-holography, she gives live concerts.
Miku Hatsune is 16 years old, with a height of 1 m 58 cm, she weighs 42 kg. The color of the hair and eyes is blue-green.
Hatsune Miku's name consists of three words: First (Hatsu), Sound (Ne), and Future (Miku). Approximately it can be translated as "The First Sound from the Future".
In order to understand all aspects of the appearance of Miku, you need to understand the fact that at first she appeared as a kind of mascot of the Vocaloid program, and only then, with increasing popularity, she went into “free swimming”, becoming a semi-independent character, nevertheless remaining , Vocaloid mascot. Now more about this.
The Vocaloid program is essentially a synthesizer of human singing. Initially, it was not popular until Crypton Future Media decided to create an official mascot for the next generation of products (a mascot is any recognizable character that personifies a school, sports team, community, military unit, event or brand). Of course, the mascot from Crypton was supposed to implement the main concept of the company - to sing. Thus, Hatsune Miku was born.
The task of coming up with an image of Miku was given to artist Kei Garo. When Kei took on Hatsune's appearance, his only guidelines were that she was an android and her color scheme should be based on the turquoise color signatures of a Yamaha synthesizer. Initially, Miku could have various hairstyles, but Kei settled on pigtails after going through various options. Crypton has officially created Miku's "passport". However, it contains only its physical and technical parameters. Crypton specialists decided not to work out Miku's personality characteristics, which allows them to create situational traits that are best suited to a particular song or concert.
Each new voice for the Vocaloid program was created based on the voices of live singers and singers. Those in front of the microphone repeated all the sounds, which the program then had to put into songs, just like putting together a mosaic of pieces. But a simple mechanical folding of sounds will not be able to give a good result and "reliable" singing.
The Vocaloid program, in addition to pairing sound fragments, can automatically or manually change many voice parameters during a song - from pitch to vibrato. Thanks to this, for example, Vocaloid is able to hit higher notes than the person who gave him his voice.
The initial sales of Vocaloid under the "patronage" of Hatsune Miku's mascot were so high that Crypton's capabilities even lagged behind demand. In the first 12 days of sales, about 3,000 copies of the program were sold. Globally, this is about one in 250 sales in the music software industry. On September 12, 2007, Amazon.co.jp reported sales of Hatsune Miku totaling 57,500,000 yen, making her the number one software sales of the time.
Despite the fact that the English version of the program never appeared, and Hatsune Miku sings only in Japanese, the singer is becoming popular in the English-speaking world. She, for example, opened for Lady Gaga during her 2014 world tour.
