Skip to content

Commit

Permalink
Order of operations matters.
Browse files Browse the repository at this point in the history
  • Loading branch information
comfyanonymous committed Aug 13, 2024
1 parent c032b11 commit 5942c17
Showing 1 changed file with 2 additions and 2 deletions.
4 changes: 2 additions & 2 deletions comfy/ldm/flux/layers.py
Original file line number Diff line number Diff line change
Expand Up @@ -170,8 +170,8 @@ def forward(self, img: Tensor, txt: Tensor, vec: Tensor, pe: Tensor):
txt_attn, img_attn = attn[:, : txt.shape[1]], attn[:, txt.shape[1] :]

# calculate the img bloks
img += img_mod1.gate * self.img_attn.proj(img_attn)
img += img_mod2.gate * self.img_mlp((1 + img_mod2.scale) * self.img_norm2(img) + img_mod2.shift)
img = img + img_mod1.gate * self.img_attn.proj(img_attn)
img = img + img_mod2.gate * self.img_mlp((1 + img_mod2.scale) * self.img_norm2(img) + img_mod2.shift)

# calculate the txt bloks
txt += txt_mod1.gate * self.txt_attn.proj(txt_attn)
Expand Down

0 comments on commit 5942c17

Please sign in to comment.