Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

参数进行梯度下降时,是否有误? #4

Open
galaxyzen opened this issue Apr 11, 2021 · 1 comment
Open

参数进行梯度下降时,是否有误? #4

galaxyzen opened this issue Apr 11, 2021 · 1 comment

Comments

@galaxyzen
Copy link

galaxyzen commented Apr 11, 2021

        if eg > 0:  # 大于0取原值,小于0则置0.即合页损失函数margin-based ranking criterion
            self.loss += eg
            temp_positive = 2 * self.learning_rate * (t - h - r)
            temp_negative = 2 * self.learning_rate * (t2 - h2 - r)
            if self.normal_form == "L1":
                temp_positive_L1 = [1 if temp_positive[i] >= 0 else -1 for i in range(self.dim)]
                temp_negative_L1 = [1 if temp_negative[i] >= 0 else -1 for i in range(self.dim)]
                temp_positive = np.array(temp_positive_L1) * self.learning_rate
                temp_negative = np.array(temp_negative_L1) * self.learning_rate

            # 对损失函数的5个参数进行梯度下降, 随机体现在sample函数上
            h += temp_positive
            t -= temp_positive
            r = r + temp_positive - temp_negative
            h2 -= temp_negative
            t2 += temp_negative

因为是随机替换head或者tail,当替换的是tail时,head没有被替换,实际上并不存在论文中所述的negative_head了,positive_head和negative_head实际上是一个东西,这种情况下在对positive_head求偏导时,是否应该包含两个部分 2 * (positve_head + relation - positive_tail) - 2 * (positive_head + relation - negative_tail)

@easternbob2019
Copy link

我认为你的理解有误。参数梯度下降是正确的。因为当我们进行首尾实体替换之后,目的是得到Negative triplet. 对于一个(t,h,r),所对应的是(t1,h1,r)。当然,有可能是t1或者是h1和正样本相同。 但是你注意看损失函数是包含5个变量.所以在进行梯度下降的时候我们都要求偏导。即使positive head 和 negative head 是相同的,但他们是不同的变量。按照你的理解的话,就不应该对h2进行梯度下降了.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants