TUNIFY GAN CODE
Now you have a code that represents a face, you can give this as input to the blended model and given that they are closely related, you will get the same face, but modified to look like a “toon” version! Below the original is on the left, and the generated image on the right (it’s hard to tell right!?) So given an example image you want to find you can find a “code” (aka latent vector) which, when given as an input to the model, will produce an output which looks almost exactly like the face you’re looking for. It’s actually a straight forward process to search for any image of a face in the model. These StyleGAN face models can produce a huge diversity of faces and it’s actually possible to find basically any face inside the model. If you generate images using the original faces model and the blended cartoon model, you can see how there is a clear relationship between the two, the identities appear to be the same, but the features have been shifted to give them a cartoon look. You end up with a hybrid which has the structure of a cartoon face, but photo realistic rendering! When Doron used my layer swapping script ( Colab here) to take the high resolution layers from the original model, and the low resolution from his fine-tuned cartoon model. So low resolution layers affect the pose of the head and shape of the face, while high resolution layers control things like lighting and texture. This gets interesting because, due to the structure of StyleGAN, different layers in the model affect the appearance in different ways. Blend the modelsīut as this model was fine tuned from the original faces model, we can perform a trick where we directly swap parts of the models around. The model tries to replicate all these things and comes off worse for it. The problem is that the style of images in the dataset is a bit of a mish-mash, some are CG some are hand drawn, and lots are quite low resolution. The output is OK for such a small amount of training on a small dataset, it’s clearly got the big eyes thing down pretty well. Once the model is trained just a little bit it gives outputs that look like below It’s only around 300 images but enough for the model to start learning what features these characters typically have. This gives good results really quickly, and even more so if the new dataset is some also of faces.ĭoron fine-tuned the faces model on a dataset of various characters from animated films.
Not everyone has multiple-GPUs and weeks of time to train a model so a shortcut lots of people is called transfer learning, where they take this “pre-trained model” and then train it on some new data. This was released with the StyleGAN2 code and paper and produces pretty fantastically high quality results. We start with the classic StyleGAN model which is trained on photos of people’s faces. So what’s going on here? This blended network is actually the product of a rather convoluted process which I’ll try and briefly explain.
TUNIFY GAN HOW TO
How to make a photo-realistic cartoon model
0 kommentar(er)
