{"id":7017,"date":"2022-12-08T16:37:52","date_gmt":"2022-12-08T13:37:52","guid":{"rendered":"http:\/\/engineering.tiu.edu.iq\/computer\/?p=7017"},"modified":"2023-06-26T10:24:42","modified_gmt":"2023-06-26T07:24:42","slug":"stable-diffusion-a-multimodal-generative-vision-language","status":"publish","type":"post","link":"https:\/\/engineering.tiu.edu.iq\/computer\/stable-diffusion-a-multimodal-generative-vision-language\/","title":{"rendered":"Seminar: Stable Diffusion: A Multimodal Generative Vision Language"},"content":{"rendered":"

Less formal than a class lecture, a seminar allows for small groups to meet and discuss academic topics or required reading, as well as set goals for research and continuing investigation.<\/p>\n

The following seminar was conducted on 80-12-2023 at 10:30 AM by Dr. Abubakar M. Ashir, Lecturer, Department of Computer Engineering Computer.<\/p>\n

Synopsis (Abstract) about the seminar:<\/p>\n

Stable Diffusion is a deep learning, text-to-image model released in 2022. It is primarily used to generate detailed images conditioned on text descriptions, though it can also be applied to other tasks such as inpainting, outpainting, and generating image-to-image translations guided by a text prompt. Stable Diffusion uses a kind of diffusion model (DM), called a latent diffusion model (LDM), diffusion models are trained with the objective of removing successive applications of Gaussian noise on training images, which can be thought of as a sequence of denoising autoencoders. Stable Diffusion consists of 3 parts: the variational autoencoder (VAE), U-Net, and an optional text encoder. The VAE encoder compresses the image from pixel space to a smaller dimensional latent space, capturing a more fundamental semantic meaning of the image. Gaussian noise is iteratively applied to the compressed latent representation during forward diffusion. The U-Net block, composed of a ResNet backbone, denoises the output from forward diffusion backwards to obtain a latent representation. Finally, the VAE decoder generates the final image by converting the representation back into pixel space. The denoising step can be flexibly conditioned on a string of text, an image, or another modality. The encoded conditioning data is exposed to denoising U-Nets via a cross-attention mechanism. For conditioning on text, the fixed, pretrained CLIP VIT-L\/14 text encoder is used to transform text prompts to an embedding space. Researchers point to increased computational efficiency for training and generation as an advantage of LDMs.<\/p>\n<\/div>

\"\"<\/a><\/div><\/div>
\"\"<\/a><\/div><\/div>
\"\"<\/a><\/div><\/div>
<\/div>
\"\"<\/a><\/div><\/div>
\"\"<\/a><\/div><\/div><\/div><\/div><\/div><\/div><\/div><\/div>\n","protected":false},"excerpt":{"rendered":"","protected":false},"author":57,"featured_media":7108,"comment_status":"closed","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"wds_primary_category":0,"footnotes":""},"categories":[43],"tags":[],"class_list":["post-7017","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-seminars"],"yoast_head":"\nSeminar: Stable Diffusion: A Multimodal Generative Vision Language - Computer Engineering Department<\/title>\n<meta name=\"robots\" content=\"index, follow, max-snippet:-1, max-image-preview:large, max-video-preview:-1\" \/>\n<link rel=\"canonical\" href=\"https:\/\/engineering.tiu.edu.iq\/computer\/stable-diffusion-a-multimodal-generative-vision-language\/\" \/>\n<meta property=\"og:locale\" content=\"en_US\" \/>\n<meta property=\"og:type\" content=\"article\" \/>\n<meta property=\"og:title\" content=\"Seminar: Stable Diffusion: A Multimodal Generative Vision Language - Computer Engineering Department\" \/>\n<meta property=\"og:url\" content=\"https:\/\/engineering.tiu.edu.iq\/computer\/stable-diffusion-a-multimodal-generative-vision-language\/\" \/>\n<meta property=\"og:site_name\" content=\"Computer Engineering Department\" \/>\n<meta property=\"article:publisher\" content=\"https:\/\/www.facebook.com\/Tishkuniversity\/?ref=bookmarks\" \/>\n<meta property=\"article:published_time\" content=\"2022-12-08T13:37:52+00:00\" \/>\n<meta property=\"article:modified_time\" content=\"2023-06-26T07:24:42+00:00\" \/>\n<meta property=\"og:image\" content=\"https:\/\/engineering.tiu.edu.iq\/computer\/wp-content\/uploads\/2022\/12\/web-posts-1-1.png\" \/>\n\t<meta property=\"og:image:width\" content=\"1920\" \/>\n\t<meta property=\"og:image:height\" content=\"1080\" \/>\n\t<meta property=\"og:image:type\" content=\"image\/png\" \/>\n<meta name=\"author\" content=\"Web Admin\" \/>\n<meta name=\"twitter:card\" content=\"summary_large_image\" \/>\n<meta name=\"twitter:creator\" content=\"@tishkuni\" \/>\n<meta name=\"twitter:site\" content=\"@tishkuni\" \/>\n<meta name=\"twitter:label1\" content=\"Written by\" \/>\n\t<meta name=\"twitter:data1\" content=\"Web Admin\" \/>\n\t<meta name=\"twitter:label2\" content=\"Est. reading time\" \/>\n\t<meta name=\"twitter:data2\" content=\"2 minutes\" \/>\n<script type=\"application\/ld+json\" class=\"yoast-schema-graph\">{\"@context\":\"https:\/\/schema.org\",\"@graph\":[{\"@type\":\"Article\",\"@id\":\"https:\/\/engineering.tiu.edu.iq\/computer\/stable-diffusion-a-multimodal-generative-vision-language\/#article\",\"isPartOf\":{\"@id\":\"https:\/\/engineering.tiu.edu.iq\/computer\/stable-diffusion-a-multimodal-generative-vision-language\/\"},\"author\":{\"name\":\"Web Admin\",\"@id\":\"https:\/\/engineering.tiu.edu.iq\/computer\/#\/schema\/person\/6950fcafea2cc5467e5114842fceb89e\"},\"headline\":\"Seminar: Stable Diffusion: A Multimodal Generative Vision Language\",\"datePublished\":\"2022-12-08T13:37:52+00:00\",\"dateModified\":\"2023-06-26T07:24:42+00:00\",\"mainEntityOfPage\":{\"@id\":\"https:\/\/engineering.tiu.edu.iq\/computer\/stable-diffusion-a-multimodal-generative-vision-language\/\"},\"wordCount\":820,\"publisher\":{\"@id\":\"https:\/\/engineering.tiu.edu.iq\/computer\/#organization\"},\"image\":{\"@id\":\"https:\/\/engineering.tiu.edu.iq\/computer\/stable-diffusion-a-multimodal-generative-vision-language\/#primaryimage\"},\"thumbnailUrl\":\"https:\/\/engineering.tiu.edu.iq\/computer\/wp-content\/uploads\/2022\/12\/web-posts-1-1.png\",\"articleSection\":[\"Seminars\"],\"inLanguage\":\"en-US\"},{\"@type\":\"WebPage\",\"@id\":\"https:\/\/engineering.tiu.edu.iq\/computer\/stable-diffusion-a-multimodal-generative-vision-language\/\",\"url\":\"https:\/\/engineering.tiu.edu.iq\/computer\/stable-diffusion-a-multimodal-generative-vision-language\/\",\"name\":\"Seminar: Stable Diffusion: A Multimodal Generative Vision Language - Computer Engineering Department\",\"isPartOf\":{\"@id\":\"https:\/\/engineering.tiu.edu.iq\/computer\/#website\"},\"primaryImageOfPage\":{\"@id\":\"https:\/\/engineering.tiu.edu.iq\/computer\/stable-diffusion-a-multimodal-generative-vision-language\/#primaryimage\"},\"image\":{\"@id\":\"https:\/\/engineering.tiu.edu.iq\/computer\/stable-diffusion-a-multimodal-generative-vision-language\/#primaryimage\"},\"thumbnailUrl\":\"https:\/\/engineering.tiu.edu.iq\/computer\/wp-content\/uploads\/2022\/12\/web-posts-1-1.png\",\"datePublished\":\"2022-12-08T13:37:52+00:00\",\"dateModified\":\"2023-06-26T07:24:42+00:00\",\"breadcrumb\":{\"@id\":\"https:\/\/engineering.tiu.edu.iq\/computer\/stable-diffusion-a-multimodal-generative-vision-language\/#breadcrumb\"},\"inLanguage\":\"en-US\",\"potentialAction\":[{\"@type\":\"ReadAction\",\"target\":[\"https:\/\/engineering.tiu.edu.iq\/computer\/stable-diffusion-a-multimodal-generative-vision-language\/\"]}]},{\"@type\":\"ImageObject\",\"inLanguage\":\"en-US\",\"@id\":\"https:\/\/engineering.tiu.edu.iq\/computer\/stable-diffusion-a-multimodal-generative-vision-language\/#primaryimage\",\"url\":\"https:\/\/engineering.tiu.edu.iq\/computer\/wp-content\/uploads\/2022\/12\/web-posts-1-1.png\",\"contentUrl\":\"https:\/\/engineering.tiu.edu.iq\/computer\/wp-content\/uploads\/2022\/12\/web-posts-1-1.png\",\"width\":1920,\"height\":1080},{\"@type\":\"BreadcrumbList\",\"@id\":\"https:\/\/engineering.tiu.edu.iq\/computer\/stable-diffusion-a-multimodal-generative-vision-language\/#breadcrumb\",\"itemListElement\":[{\"@type\":\"ListItem\",\"position\":1,\"name\":\"Home\",\"item\":\"https:\/\/engineering.tiu.edu.iq\/computer\/\"},{\"@type\":\"ListItem\",\"position\":2,\"name\":\"Seminar: Stable Diffusion: A Multimodal Generative Vision Language\"}]},{\"@type\":\"WebSite\",\"@id\":\"https:\/\/engineering.tiu.edu.iq\/computer\/#website\",\"url\":\"https:\/\/engineering.tiu.edu.iq\/computer\/\",\"name\":\"TIU\",\"description\":\"The Future is Here\",\"publisher\":{\"@id\":\"https:\/\/engineering.tiu.edu.iq\/computer\/#organization\"},\"potentialAction\":[{\"@type\":\"SearchAction\",\"target\":{\"@type\":\"EntryPoint\",\"urlTemplate\":\"https:\/\/engineering.tiu.edu.iq\/computer\/?s={search_term_string}\"},\"query-input\":{\"@type\":\"PropertyValueSpecification\",\"valueRequired\":true,\"valueName\":\"search_term_string\"}}],\"inLanguage\":\"en-US\"},{\"@type\":\"Organization\",\"@id\":\"https:\/\/engineering.tiu.edu.iq\/computer\/#organization\",\"name\":\"TIU\",\"url\":\"https:\/\/engineering.tiu.edu.iq\/computer\/\",\"logo\":{\"@type\":\"ImageObject\",\"inLanguage\":\"en-US\",\"@id\":\"https:\/\/engineering.tiu.edu.iq\/computer\/#\/schema\/logo\/image\/\",\"url\":\"\",\"contentUrl\":\"\",\"caption\":\"TIU\"},\"image\":{\"@id\":\"https:\/\/engineering.tiu.edu.iq\/computer\/#\/schema\/logo\/image\/\"},\"sameAs\":[\"https:\/\/www.facebook.com\/Tishkuniversity\/?ref=bookmarks\",\"https:\/\/x.com\/tishkuni\"]},{\"@type\":\"Person\",\"@id\":\"https:\/\/engineering.tiu.edu.iq\/computer\/#\/schema\/person\/6950fcafea2cc5467e5114842fceb89e\",\"name\":\"Web Admin\",\"image\":{\"@type\":\"ImageObject\",\"inLanguage\":\"en-US\",\"@id\":\"https:\/\/secure.gravatar.com\/avatar\/3a47b9231d151155cd33bd25d059ecb52841805eb3d30eec2e06be3ffe6ee961?s=96&d=mm&r=g\",\"url\":\"https:\/\/secure.gravatar.com\/avatar\/3a47b9231d151155cd33bd25d059ecb52841805eb3d30eec2e06be3ffe6ee961?s=96&d=mm&r=g\",\"contentUrl\":\"https:\/\/secure.gravatar.com\/avatar\/3a47b9231d151155cd33bd25d059ecb52841805eb3d30eec2e06be3ffe6ee961?s=96&d=mm&r=g\",\"caption\":\"Web Admin\"},\"url\":\"https:\/\/engineering.tiu.edu.iq\/computer\/author\/web_admin\/\"}]}<\/script>\n<!-- \/ Yoast SEO plugin. -->","yoast_head_json":{"title":"Seminar: Stable Diffusion: A Multimodal Generative Vision Language - Computer Engineering Department","robots":{"index":"index","follow":"follow","max-snippet":"max-snippet:-1","max-image-preview":"max-image-preview:large","max-video-preview":"max-video-preview:-1"},"canonical":"https:\/\/engineering.tiu.edu.iq\/computer\/stable-diffusion-a-multimodal-generative-vision-language\/","og_locale":"en_US","og_type":"article","og_title":"Seminar: Stable Diffusion: A Multimodal Generative Vision Language - Computer Engineering Department","og_url":"https:\/\/engineering.tiu.edu.iq\/computer\/stable-diffusion-a-multimodal-generative-vision-language\/","og_site_name":"Computer Engineering Department","article_publisher":"https:\/\/www.facebook.com\/Tishkuniversity\/?ref=bookmarks","article_published_time":"2022-12-08T13:37:52+00:00","article_modified_time":"2023-06-26T07:24:42+00:00","og_image":[{"width":1920,"height":1080,"url":"https:\/\/engineering.tiu.edu.iq\/computer\/wp-content\/uploads\/2022\/12\/web-posts-1-1.png","type":"image\/png"}],"author":"Web Admin","twitter_card":"summary_large_image","twitter_creator":"@tishkuni","twitter_site":"@tishkuni","twitter_misc":{"Written by":"Web Admin","Est. reading time":"2 minutes"},"schema":{"@context":"https:\/\/schema.org","@graph":[{"@type":"Article","@id":"https:\/\/engineering.tiu.edu.iq\/computer\/stable-diffusion-a-multimodal-generative-vision-language\/#article","isPartOf":{"@id":"https:\/\/engineering.tiu.edu.iq\/computer\/stable-diffusion-a-multimodal-generative-vision-language\/"},"author":{"name":"Web Admin","@id":"https:\/\/engineering.tiu.edu.iq\/computer\/#\/schema\/person\/6950fcafea2cc5467e5114842fceb89e"},"headline":"Seminar: Stable Diffusion: A Multimodal Generative Vision Language","datePublished":"2022-12-08T13:37:52+00:00","dateModified":"2023-06-26T07:24:42+00:00","mainEntityOfPage":{"@id":"https:\/\/engineering.tiu.edu.iq\/computer\/stable-diffusion-a-multimodal-generative-vision-language\/"},"wordCount":820,"publisher":{"@id":"https:\/\/engineering.tiu.edu.iq\/computer\/#organization"},"image":{"@id":"https:\/\/engineering.tiu.edu.iq\/computer\/stable-diffusion-a-multimodal-generative-vision-language\/#primaryimage"},"thumbnailUrl":"https:\/\/engineering.tiu.edu.iq\/computer\/wp-content\/uploads\/2022\/12\/web-posts-1-1.png","articleSection":["Seminars"],"inLanguage":"en-US"},{"@type":"WebPage","@id":"https:\/\/engineering.tiu.edu.iq\/computer\/stable-diffusion-a-multimodal-generative-vision-language\/","url":"https:\/\/engineering.tiu.edu.iq\/computer\/stable-diffusion-a-multimodal-generative-vision-language\/","name":"Seminar: Stable Diffusion: A Multimodal Generative Vision Language - Computer Engineering Department","isPartOf":{"@id":"https:\/\/engineering.tiu.edu.iq\/computer\/#website"},"primaryImageOfPage":{"@id":"https:\/\/engineering.tiu.edu.iq\/computer\/stable-diffusion-a-multimodal-generative-vision-language\/#primaryimage"},"image":{"@id":"https:\/\/engineering.tiu.edu.iq\/computer\/stable-diffusion-a-multimodal-generative-vision-language\/#primaryimage"},"thumbnailUrl":"https:\/\/engineering.tiu.edu.iq\/computer\/wp-content\/uploads\/2022\/12\/web-posts-1-1.png","datePublished":"2022-12-08T13:37:52+00:00","dateModified":"2023-06-26T07:24:42+00:00","breadcrumb":{"@id":"https:\/\/engineering.tiu.edu.iq\/computer\/stable-diffusion-a-multimodal-generative-vision-language\/#breadcrumb"},"inLanguage":"en-US","potentialAction":[{"@type":"ReadAction","target":["https:\/\/engineering.tiu.edu.iq\/computer\/stable-diffusion-a-multimodal-generative-vision-language\/"]}]},{"@type":"ImageObject","inLanguage":"en-US","@id":"https:\/\/engineering.tiu.edu.iq\/computer\/stable-diffusion-a-multimodal-generative-vision-language\/#primaryimage","url":"https:\/\/engineering.tiu.edu.iq\/computer\/wp-content\/uploads\/2022\/12\/web-posts-1-1.png","contentUrl":"https:\/\/engineering.tiu.edu.iq\/computer\/wp-content\/uploads\/2022\/12\/web-posts-1-1.png","width":1920,"height":1080},{"@type":"BreadcrumbList","@id":"https:\/\/engineering.tiu.edu.iq\/computer\/stable-diffusion-a-multimodal-generative-vision-language\/#breadcrumb","itemListElement":[{"@type":"ListItem","position":1,"name":"Home","item":"https:\/\/engineering.tiu.edu.iq\/computer\/"},{"@type":"ListItem","position":2,"name":"Seminar: Stable Diffusion: A Multimodal Generative Vision Language"}]},{"@type":"WebSite","@id":"https:\/\/engineering.tiu.edu.iq\/computer\/#website","url":"https:\/\/engineering.tiu.edu.iq\/computer\/","name":"TIU","description":"The Future is Here","publisher":{"@id":"https:\/\/engineering.tiu.edu.iq\/computer\/#organization"},"potentialAction":[{"@type":"SearchAction","target":{"@type":"EntryPoint","urlTemplate":"https:\/\/engineering.tiu.edu.iq\/computer\/?s={search_term_string}"},"query-input":{"@type":"PropertyValueSpecification","valueRequired":true,"valueName":"search_term_string"}}],"inLanguage":"en-US"},{"@type":"Organization","@id":"https:\/\/engineering.tiu.edu.iq\/computer\/#organization","name":"TIU","url":"https:\/\/engineering.tiu.edu.iq\/computer\/","logo":{"@type":"ImageObject","inLanguage":"en-US","@id":"https:\/\/engineering.tiu.edu.iq\/computer\/#\/schema\/logo\/image\/","url":"","contentUrl":"","caption":"TIU"},"image":{"@id":"https:\/\/engineering.tiu.edu.iq\/computer\/#\/schema\/logo\/image\/"},"sameAs":["https:\/\/www.facebook.com\/Tishkuniversity\/?ref=bookmarks","https:\/\/x.com\/tishkuni"]},{"@type":"Person","@id":"https:\/\/engineering.tiu.edu.iq\/computer\/#\/schema\/person\/6950fcafea2cc5467e5114842fceb89e","name":"Web Admin","image":{"@type":"ImageObject","inLanguage":"en-US","@id":"https:\/\/secure.gravatar.com\/avatar\/3a47b9231d151155cd33bd25d059ecb52841805eb3d30eec2e06be3ffe6ee961?s=96&d=mm&r=g","url":"https:\/\/secure.gravatar.com\/avatar\/3a47b9231d151155cd33bd25d059ecb52841805eb3d30eec2e06be3ffe6ee961?s=96&d=mm&r=g","contentUrl":"https:\/\/secure.gravatar.com\/avatar\/3a47b9231d151155cd33bd25d059ecb52841805eb3d30eec2e06be3ffe6ee961?s=96&d=mm&r=g","caption":"Web Admin"},"url":"https:\/\/engineering.tiu.edu.iq\/computer\/author\/web_admin\/"}]}},"_links":{"self":[{"href":"https:\/\/engineering.tiu.edu.iq\/computer\/wp-json\/wp\/v2\/posts\/7017","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/engineering.tiu.edu.iq\/computer\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/engineering.tiu.edu.iq\/computer\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/engineering.tiu.edu.iq\/computer\/wp-json\/wp\/v2\/users\/57"}],"replies":[{"embeddable":true,"href":"https:\/\/engineering.tiu.edu.iq\/computer\/wp-json\/wp\/v2\/comments?post=7017"}],"version-history":[{"count":4,"href":"https:\/\/engineering.tiu.edu.iq\/computer\/wp-json\/wp\/v2\/posts\/7017\/revisions"}],"predecessor-version":[{"id":7109,"href":"https:\/\/engineering.tiu.edu.iq\/computer\/wp-json\/wp\/v2\/posts\/7017\/revisions\/7109"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/engineering.tiu.edu.iq\/computer\/wp-json\/wp\/v2\/media\/7108"}],"wp:attachment":[{"href":"https:\/\/engineering.tiu.edu.iq\/computer\/wp-json\/wp\/v2\/media?parent=7017"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/engineering.tiu.edu.iq\/computer\/wp-json\/wp\/v2\/categories?post=7017"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/engineering.tiu.edu.iq\/computer\/wp-json\/wp\/v2\/tags?post=7017"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}