Commit 0cf2f6f6 authored by Simon Guo's avatar Simon Guo Committed by Linus Torvalds

mm: mlock: check against vma for actual mlock() size

In do_mlock(), the check against locked memory limitation has a hole
which will fail following cases at step 3):

 1) User has a memory chunk from addressA with 50k, and user mem lock
    rlimit is 64k.
 2) mlock(addressA, 30k)
 3) mlock(addressA, 40k)

The 3rd step should have been allowed since the 40k request is
intersected with the previous 30k at step 2), and the 3rd step is
actually for mlock on the extra 10k memory.

This patch checks vma to caculate the actual "new" mlock size, if
necessary, and ajust the logic to fix this issue.

[ clean up comment layout]
[ correct a typo in count_mm_mlocked_page_nr()]
Link: default avatarSimon Guo <>
Cc: Alexey Klimov <>
Cc: Eric B Munson <>
Cc: Geert Uytterhoeven <>
Cc: "Kirill A. Shutemov" <>
Cc: Mel Gorman <>
Cc: Michal Hocko <>
Cc: Shuah Khan <>
Cc: Simon Guo <>
Cc: Thierry Reding <>
Cc: Vlastimil Babka <>
Cc: David Rientjes <>
Signed-off-by: default avatarAndrew Morton <>
Signed-off-by: default avatarLinus Torvalds <>
parent 9254990f
......@@ -617,6 +617,45 @@ static int apply_vma_lock_flags(unsigned long start, size_t len,
return error;
* Go through vma areas and sum size of mlocked
* vma pages, as return value.
* Note deferred memory locking case(mlock2(,,MLOCK_ONFAULT)
* is also counted.
* Return value: previously mlocked page counts
static int count_mm_mlocked_page_nr(struct mm_struct *mm,
unsigned long start, size_t len)
struct vm_area_struct *vma;
int count = 0;
if (mm == NULL)
mm = current->mm;
vma = find_vma(mm, start);
if (vma == NULL)
vma = mm->mmap;
for (; vma ; vma = vma->vm_next) {
if (start >= vma->vm_end)
if (start + len <= vma->vm_start)
if (vma->vm_flags & VM_LOCKED) {
if (start > vma->vm_start)
count -= (start - vma->vm_start);
if (start + len < vma->vm_end) {
count += start + len - vma->vm_start;
count += vma->vm_end - vma->vm_start;
return count >> PAGE_SHIFT;
static __must_check int do_mlock(unsigned long start, size_t len, vm_flags_t flags)
unsigned long locked;
......@@ -639,6 +678,16 @@ static __must_check int do_mlock(unsigned long start, size_t len, vm_flags_t fla
return -EINTR;
locked += current->mm->locked_vm;
if ((locked > lock_limit) && (!capable(CAP_IPC_LOCK))) {
* It is possible that the regions requested intersect with
* previously mlocked areas, that part area in "mm->locked_vm"
* should not be counted to new mlock increment count. So check
* and adjust locked count if necessary.
locked -= count_mm_mlocked_page_nr(current->mm,
start, len);
/* check against resource limits */
if ((locked <= lock_limit) || capable(CAP_IPC_LOCK))
Markdown is supported
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment