The do_tcp_setsockopt function in net/ipv4/tcp.c in the Linux kernel before 2.6.37-rc2 does not properly restrict TCP_MAXSEG (aka MSS) values, which allows local users to cause a denial of service (OOPS) via a setsockopt call that specifies a small value, leading to a divide-by-zero error or incorrect use of a signed integer.

References

• https://nvd.nist.gov/vuln/detail/CVE-2010-4165
• https://bugzilla.redhat.com/show_bug.cgi?id=652508
• http://git.kernel.org/?p=linux/kernel/git/torvalds/linux-2.6.git;a=commit;h=7a1abd08d52fdeddb3e9a5a33f2f15cc6a5674d2
• http://lists.opensuse.org/opensuse-security-announce/2011-01/msg00000.html
• http://lists.opensuse.org/opensuse-security-announce/2011-01/msg00001.html
• http://lists.opensuse.org/opensuse-security-announce/2011-01/msg00004.html
• http://lists.opensuse.org/opensuse-security-announce/2011-02/msg00000.html
• http://secunia.com/advisories/42778
• http://secunia.com/advisories/42801
• http://secunia.com/advisories/42932
• http://securityreason.com/securityalert/8111
• http://securityreason.com/securityalert/8123
• http://www.kernel.org/pub/linux/kernel/v2.6/testing/ChangeLog-2.6.37-rc2
• http://www.mandriva.com/security/advisories?name=MDVSA-2011:029
• http://www.mandriva.com/security/advisories?name=MDVSA-2011:051
• http://www.openwall.com/lists/oss-security/2010/11/12/1
• http://www.openwall.com/lists/oss-security/2010/11/12/4
• http://www.osvdb.org/69241
• http://www.securityfocus.com/bid/44830
• http://www.spinics.net/lists/netdev/msg146405.html
• http://www.spinics.net/lists/netdev/msg146495.html
• http://www.vupen.com/english/advisories/2011/0012
• http://www.vupen.com/english/advisories/2011/0124
• http://www.vupen.com/english/advisories/2011/0298
• http://git.kernel.org/?p=linux/kernel/git/torvalds/linux-2.6.git%3Ba=commit%3Bh=7a1abd08d52fdeddb3e9a5a33f2f15cc6a5674d2