Address sanitizer memory usage

classic Classic list List threaded Threaded
6 messages Options
Reply | Threaded
Open this post in threaded view
|

Address sanitizer memory usage

Eric Fiselier via cfe-dev
Hi,

I'm working on an iPhone app that rarely runs out of memory normally, but with the address sanitizer on will crash after a couple minutes. I don't remember it crashing this much with Xcode 8's version of clang, but in any case I'm using Xcode 9 now. Although crashing after a couple minutes is fine for testing during development, I would like to distribute the app to internal testers with asan built into it, and I don't want to impact their experience so much. Is there any simple way to reduce memory use, e.g. by turning off some component of asan?

Thanks,
Michael

_______________________________________________
cfe-dev mailing list
[hidden email]
http://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-dev
Reply | Threaded
Open this post in threaded view
|

Re: Address sanitizer memory usage

Eric Fiselier via cfe-dev
+ Devin and Kuba from Apple, who may have iOS-specific knowledge. 

We've recently made several changes that make asan consume less memory, but
* these changes are not yet in Xcode 9 (I assume)
* I am not 100% sure these changes affect iOS (we tested only on Linux, Kuba has recently applied related patches for Mac)
* Your problem might be caused by something else

Typically the biggest source of memory consumption is quarantine and the stack trace storage (stack_depot).
If you are able to set ASAN_OPTIONS env var, please try this: 
ASAN_OPTIONS=quarantine_size_mb=1:malloc_context_size=5

quarantine_size_mb=1 limits the quarantine size (and thus reduces the ability to find use-after-free)
malloc_context_size=5 reduces the length of stack traces that asan stores for future bug reporting 

If these don't help, we'll need more details (e.g. the iOS's analog of /proc/PID/maps for the process at the time of failure)

--kcc 



On Thu, Dec 7, 2017 at 12:56 PM, Michael Eisel via cfe-dev <[hidden email]> wrote:
Hi,

I'm working on an iPhone app that rarely runs out of memory normally, but with the address sanitizer on will crash after a couple minutes. I don't remember it crashing this much with Xcode 8's version of clang, but in any case I'm using Xcode 9 now. Although crashing after a couple minutes is fine for testing during development, I would like to distribute the app to internal testers with asan built into it, and I don't want to impact their experience so much. Is there any simple way to reduce memory use, e.g. by turning off some component of asan?

Thanks,
Michael

_______________________________________________
cfe-dev mailing list
[hidden email]
http://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-dev



_______________________________________________
cfe-dev mailing list
[hidden email]
http://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-dev
Reply | Threaded
Open this post in threaded view
|

Re: Address sanitizer memory usage

Eric Fiselier via cfe-dev
Hi, so when you say "ASAN_OPTIONS=quarantine_size_mb=1:malloc_context_size=5", do you mean that it's one key-value pair with key "ASAN_OPTIONS" and value "quarantine_size_mb=1:malloc_context_size=5" ? I tried that, but unfortunately, the phone still crashes after a few minutes of typical usage. Here's the trace:

==1020==ERROR: AddressSanitizer failed to allocate 0x80000 (524288) bytes of SetAlternateSignalStack (error code: 12)
==1020==Process memory map follows:
  0x000195805000-0x000195819000  /System/Library/PrivateFrameworks/EmojiFoundation.framework/EmojiFoundation
  0x0001b216e458-0x0001b216ee98  /System/Library/PrivateFrameworks/EmojiFoundation.framework/EmojiFoundation
  0x0001ae3f0178-0x0001ae3ff328  /System/Library/PrivateFrameworks/EmojiFoundation.framework/EmojiFoundation
  0x0001b315cf10-0x0001b315d070  /System/Library/PrivateFrameworks/EmojiFoundation.framework/EmojiFoundation
  0x0001b5366000-0x0001baa0c000  /System/Library/PrivateFrameworks/EmojiFoundation.framework/EmojiFoundation
  0x0001937e6000-0x0001937f8000  /System/Library/PrivateFrameworks/CoreFollowUp.framework/CoreFollowUp
  0x0001b20576f0-0x0001b2058280  /System/Library/PrivateFrameworks/CoreFollowUp.framework/CoreFollowUp
  0x0001add00b60-0x0001add06638  /System/Library/PrivateFrameworks/CoreFollowUp.framework/CoreFollowUp
  0x0001b31439a8-0x0001b3143c78  /System/Library/PrivateFrameworks/CoreFollowUp.framework/CoreFollowUp
  0x0001b5366000-0x0001baa0c000  /System/Library/PrivateFrameworks/CoreFollowUp.framework/CoreFollowUp
  0x00019447f000-0x00019448b000  /System/Library/PrivateFrameworks/SetupAssistantSupport.framework/SetupAssistantSupport
  0x0001b20bd288-0x0001b20be258  /System/Library/PrivateFrameworks/SetupAssistantSupport.framework/SetupAssistantSupport
  0x0001adfc1300-0x0001adfc4b38  /System/Library/PrivateFrameworks/SetupAssistantSupport.framework/SetupAssistantSupport
  0x0001b5366000-0x0001baa0c000  /System/Library/PrivateFrameworks/SetupAssistantSupport.framework/SetupAssistantSupport
  0x00019448b000-0x0001944b3000  /System/Library/PrivateFrameworks/SetupAssistant.framework/SetupAssistant
  0x0001b20be258-0x0001b20bfbc8  /System/Library/PrivateFrameworks/SetupAssistant.framework/SetupAssistant
  0x0001adfc4b38-0x0001adfcb630  /System/Library/PrivateFrameworks/SetupAssistant.framework/SetupAssistant
  0x0001b30ff0f0-0x0001b30ff5d8  /System/Library/PrivateFrameworks/SetupAssistant.framework/SetupAssistant
  0x0001b5366000-0x0001baa0c000  /System/Library/PrivateFrameworks/SetupAssistant.framework/SetupAssistant
  0x00019511e000-0x000195138000  /System/Library/PrivateFrameworks/CoreSDB.framework/CoreSDB
  0x0001b2121128-0x0001b2121540  /System/Library/PrivateFrameworks/CoreSDB.framework/CoreSDB
  0x0001ae21d1b8-0x0001ae21f770  /System/Library/PrivateFrameworks/CoreSDB.framework/CoreSDB
  0x0001b5366000-0x0001baa0c000  /System/Library/PrivateFrameworks/CoreSDB.framework/CoreSDB
  0x000195819000-0x00019592c000  /System/Library/PrivateFrameworks/IMDPersistence.framework/IMDPersistence
  0x0001b216ee98-0x0001b2170ae8  /System/Library/PrivateFrameworks/IMDPersistence.framework/IMDPersistence
2017-12-12 16:18:59.529 [tid=189434][general][info] Download Manager --- Request is dispatched to download manager --- request key is /bq/update_events-A0F8564D-26E2-45C2-A1B7-D540A2BFE249
  0x0001b3151788-0x0001b31517d8  /System/Library/PrivateFrameworks/IMDPersistence.framework/IMDPersistence
  0x0001ae3ff328-0x0001ae4215b0  /System/Library/PrivateFrameworks/IMDPersistence.framework/IMDPersistence
  0x0001b5366000-0x0001baa0c000  /System/Library/PrivateFrameworks/IMDPersistence.framework/IMDPersistence
  0x000195138000-0x00019518e000  /System/Library/PrivateFrameworks/IMSharedUtilities.framework/IMSharedUtilities
  0x0001b2121540-0x0001b21256c8  /System/Library/PrivateFrameworks/IMSharedUtilities.framework/IMSharedUtilities
  0x0001ae21f770-0x0001ae239ad8  /System/Library/PrivateFrameworks/IMSharedUtilities.framework/IMSharedUtilities
  0x0001b312bf50-0x0001b312bff0  /System/Library/PrivateFrameworks/IMSharedUtilities.framework/IMSharedUtilities
  0x0001b5366000-0x0001baa0c000  /System/Library/PrivateFrameworks/IMSharedUtilities.framework/IMSharedUtilities
  0x000196745000-0x0001968e4000  /System/Library/PrivateFrameworks/IMCore.framework/IMCore
  0x0001b21f60c8-0x0001b21ff2e1  /System/Library/PrivateFrameworks/IMCore.framework/IMCore
  0x0001ae607b68-0x0001ae648888  /System/Library/PrivateFrameworks/IMCore.framework/IMCore
  0x0001b31352b0-0x0001b3135cc0  /System/Library/PrivateFrameworks/IMCore.framework/IMCore
  0x0001b5366000-0x0001baa0c000  /System/Library/PrivateFrameworks/IMCore.framework/IMCore
  0x00019868f000-0x0001986ec000  /System/Library/PrivateFrameworks/IMAVCore.framework/IMAVCore
  0x0001b23195d0-0x0001b231bf70  /System/Library/PrivateFrameworks/IMAVCore.framework/IMAVCore
  0x0001aec98888-0x0001aeca8ae8  /System/Library/PrivateFrameworks/IMAVCore.framework/IMAVCore
  0x0001b3158e28-0x0001b3158fe0  /System/Library/PrivateFrameworks/IMAVCore.framework/IMAVCore
  0x0001b5366000-0x0001baa0c000  /System/Library/PrivateFrameworks/IMAVCore.framework/IMAVCore
  0x0001a7670000-0x0001a7676000  /System/Library/PrivateFrameworks/TextInputUI.framework/TextInputUI
  0x0001b2f184e0-0x0001b2f18a38  /System/Library/PrivateFrameworks/TextInputUI.framework/TextInputUI
  0x0001b31c02a8-0x0001b31c0358  /System/Library/PrivateFrameworks/TextInputUI.framework/TextInputUI
  0x0001b1211ac8-0x0001b1212690  /System/Library/PrivateFrameworks/TextInputUI.framework/TextInputUI
  0x0001b5366000-0x0001baa0c000  /System/Library/PrivateFrameworks/TextInputUI.framework/TextInputUI
  0x0001a39f1000-0x0001a3a08000  /System/Library/PrivateFrameworks/CoreAccessories.framework/CoreAccessories
  0x0001b2c9e418-0x0001b2c9f070  /System/Library/PrivateFrameworks/CoreAccessories.framework/CoreAccessories
  0x0001b06229b0-0x0001b06282b0  /System/Library/PrivateFrameworks/CoreAccessories.framework/CoreAccessories
  0x0001b3150a98-0x0001b3150b81  /System/Library/PrivateFrameworks/CoreAccessories.framework/CoreAccessories
2017-12-12 16:18:59.529 [tid=188882][general][info] Download Manager --- RequestTask is created at download manager ---  request key is /bq/update_events-A0F8564D-26E2-45C2-A1B7-D540A2BFE249, dispatch_to_task_create delay is 0
  0x0001b5366000-0x0001baa0c000  /System/Library/PrivateFrameworks/CoreAccessories.framework/CoreAccessories
  0x0001a1361000-0x0001a1616000  /System/Library/CoreServices/RawCamera.bundle/RawCamera
  0x0001b2ad2850-0x0001b2aeb838  /System/Library/CoreServices/RawCamera.bundle/RawCamera
  0x0001b02b2360-0x0001b02f97d0  /System/Library/CoreServices/RawCamera.bundle/RawCamera
ERROR: Failed to mmap
Warning: hit breakpoint while running function, skipping commands and conditions to prevent recursion.warning: could not execute support code to read Objective-C class data in the process. This may reduce the quality of type information available.
AddressSanitizer report breakpoint hit. Use 'thread info -s' to get extended information about the report.


On Fri, Dec 8, 2017 at 10:12 AM, Kostya Serebryany <[hidden email]> wrote:
+ Devin and Kuba from Apple, who may have iOS-specific knowledge. 

We've recently made several changes that make asan consume less memory, but
* these changes are not yet in Xcode 9 (I assume)
* I am not 100% sure these changes affect iOS (we tested only on Linux, Kuba has recently applied related patches for Mac)
* Your problem might be caused by something else

Typically the biggest source of memory consumption is quarantine and the stack trace storage (stack_depot).
If you are able to set ASAN_OPTIONS env var, please try this: 
ASAN_OPTIONS=quarantine_size_mb=1:malloc_context_size=5

quarantine_size_mb=1 limits the quarantine size (and thus reduces the ability to find use-after-free)
malloc_context_size=5 reduces the length of stack traces that asan stores for future bug reporting 

If these don't help, we'll need more details (e.g. the iOS's analog of /proc/PID/maps for the process at the time of failure)

--kcc 



On Thu, Dec 7, 2017 at 12:56 PM, Michael Eisel via cfe-dev <[hidden email]> wrote:
Hi,

I'm working on an iPhone app that rarely runs out of memory normally, but with the address sanitizer on will crash after a couple minutes. I don't remember it crashing this much with Xcode 8's version of clang, but in any case I'm using Xcode 9 now. Although crashing after a couple minutes is fine for testing during development, I would like to distribute the app to internal testers with asan built into it, and I don't want to impact their experience so much. Is there any simple way to reduce memory use, e.g. by turning off some component of asan?

Thanks,
Michael

_______________________________________________
cfe-dev mailing list
[hidden email]
http://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-dev




_______________________________________________
cfe-dev mailing list
[hidden email]
http://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-dev
Reply | Threaded
Open this post in threaded view
|

Re: Address sanitizer memory usage

Eric Fiselier via cfe-dev
Hi Michael,

Using an app under ASan is expected to use 2x or 3x more memory. I don't think think any of the available tweaks will get you below a ~2x overhead. Is it the case that your app is already using hundreds of MBs of memory without ASan? If yes, then with ASan you are simply running out of available physical memory of the device and I don't think there's anything you can do.

If you're seeing a much larger memory overhead by ASan (>5x), then that's likely a bug in ASan and I'd like to know some details (ideally a small reproducer).

Kuba

On Dec 12, 2017, at 4:23 PM, Michael Eisel <[hidden email]> wrote:

Hi, so when you say "ASAN_OPTIONS=quarantine_size_mb=1:malloc_context_size=5", do you mean that it's one key-value pair with key "ASAN_OPTIONS" and value "quarantine_size_mb=1:malloc_context_size=5" ? I tried that, but unfortunately, the phone still crashes after a few minutes of typical usage. Here's the trace:

==1020==ERROR: AddressSanitizer failed to allocate 0x80000 (524288) bytes of SetAlternateSignalStack (error code: 12)
==1020==Process memory map follows:
  0x000195805000-0x000195819000  /System/Library/PrivateFrameworks/EmojiFoundation.framework/EmojiFoundation
  0x0001b216e458-0x0001b216ee98  /System/Library/PrivateFrameworks/EmojiFoundation.framework/EmojiFoundation
  0x0001ae3f0178-0x0001ae3ff328  /System/Library/PrivateFrameworks/EmojiFoundation.framework/EmojiFoundation
  0x0001b315cf10-0x0001b315d070  /System/Library/PrivateFrameworks/EmojiFoundation.framework/EmojiFoundation
  0x0001b5366000-0x0001baa0c000  /System/Library/PrivateFrameworks/EmojiFoundation.framework/EmojiFoundation
  0x0001937e6000-0x0001937f8000  /System/Library/PrivateFrameworks/CoreFollowUp.framework/CoreFollowUp
  0x0001b20576f0-0x0001b2058280  /System/Library/PrivateFrameworks/CoreFollowUp.framework/CoreFollowUp
  0x0001add00b60-0x0001add06638  /System/Library/PrivateFrameworks/CoreFollowUp.framework/CoreFollowUp
  0x0001b31439a8-0x0001b3143c78  /System/Library/PrivateFrameworks/CoreFollowUp.framework/CoreFollowUp
  0x0001b5366000-0x0001baa0c000  /System/Library/PrivateFrameworks/CoreFollowUp.framework/CoreFollowUp
  0x00019447f000-0x00019448b000  /System/Library/PrivateFrameworks/SetupAssistantSupport.framework/SetupAssistantSupport
  0x0001b20bd288-0x0001b20be258  /System/Library/PrivateFrameworks/SetupAssistantSupport.framework/SetupAssistantSupport
  0x0001adfc1300-0x0001adfc4b38  /System/Library/PrivateFrameworks/SetupAssistantSupport.framework/SetupAssistantSupport
  0x0001b5366000-0x0001baa0c000  /System/Library/PrivateFrameworks/SetupAssistantSupport.framework/SetupAssistantSupport
  0x00019448b000-0x0001944b3000  /System/Library/PrivateFrameworks/SetupAssistant.framework/SetupAssistant
  0x0001b20be258-0x0001b20bfbc8  /System/Library/PrivateFrameworks/SetupAssistant.framework/SetupAssistant
  0x0001adfc4b38-0x0001adfcb630  /System/Library/PrivateFrameworks/SetupAssistant.framework/SetupAssistant
  0x0001b30ff0f0-0x0001b30ff5d8  /System/Library/PrivateFrameworks/SetupAssistant.framework/SetupAssistant
  0x0001b5366000-0x0001baa0c000  /System/Library/PrivateFrameworks/SetupAssistant.framework/SetupAssistant
  0x00019511e000-0x000195138000  /System/Library/PrivateFrameworks/CoreSDB.framework/CoreSDB
  0x0001b2121128-0x0001b2121540  /System/Library/PrivateFrameworks/CoreSDB.framework/CoreSDB
  0x0001ae21d1b8-0x0001ae21f770  /System/Library/PrivateFrameworks/CoreSDB.framework/CoreSDB
  0x0001b5366000-0x0001baa0c000  /System/Library/PrivateFrameworks/CoreSDB.framework/CoreSDB
  0x000195819000-0x00019592c000  /System/Library/PrivateFrameworks/IMDPersistence.framework/IMDPersistence
  0x0001b216ee98-0x0001b2170ae8  /System/Library/PrivateFrameworks/IMDPersistence.framework/IMDPersistence
2017-12-12 16:18:59.529 [tid=189434][general][info] Download Manager --- Request is dispatched to download manager --- request key is /bq/update_events-A0F8564D-26E2-45C2-A1B7-D540A2BFE249
  0x0001b3151788-0x0001b31517d8  /System/Library/PrivateFrameworks/IMDPersistence.framework/IMDPersistence
  0x0001ae3ff328-0x0001ae4215b0  /System/Library/PrivateFrameworks/IMDPersistence.framework/IMDPersistence
  0x0001b5366000-0x0001baa0c000  /System/Library/PrivateFrameworks/IMDPersistence.framework/IMDPersistence
  0x000195138000-0x00019518e000  /System/Library/PrivateFrameworks/IMSharedUtilities.framework/IMSharedUtilities
  0x0001b2121540-0x0001b21256c8  /System/Library/PrivateFrameworks/IMSharedUtilities.framework/IMSharedUtilities
  0x0001ae21f770-0x0001ae239ad8  /System/Library/PrivateFrameworks/IMSharedUtilities.framework/IMSharedUtilities
  0x0001b312bf50-0x0001b312bff0  /System/Library/PrivateFrameworks/IMSharedUtilities.framework/IMSharedUtilities
  0x0001b5366000-0x0001baa0c000  /System/Library/PrivateFrameworks/IMSharedUtilities.framework/IMSharedUtilities
  0x000196745000-0x0001968e4000  /System/Library/PrivateFrameworks/IMCore.framework/IMCore
  0x0001b21f60c8-0x0001b21ff2e1  /System/Library/PrivateFrameworks/IMCore.framework/IMCore
  0x0001ae607b68-0x0001ae648888  /System/Library/PrivateFrameworks/IMCore.framework/IMCore
  0x0001b31352b0-0x0001b3135cc0  /System/Library/PrivateFrameworks/IMCore.framework/IMCore
  0x0001b5366000-0x0001baa0c000  /System/Library/PrivateFrameworks/IMCore.framework/IMCore
  0x00019868f000-0x0001986ec000  /System/Library/PrivateFrameworks/IMAVCore.framework/IMAVCore
  0x0001b23195d0-0x0001b231bf70  /System/Library/PrivateFrameworks/IMAVCore.framework/IMAVCore
  0x0001aec98888-0x0001aeca8ae8  /System/Library/PrivateFrameworks/IMAVCore.framework/IMAVCore
  0x0001b3158e28-0x0001b3158fe0  /System/Library/PrivateFrameworks/IMAVCore.framework/IMAVCore
  0x0001b5366000-0x0001baa0c000  /System/Library/PrivateFrameworks/IMAVCore.framework/IMAVCore
  0x0001a7670000-0x0001a7676000  /System/Library/PrivateFrameworks/TextInputUI.framework/TextInputUI
  0x0001b2f184e0-0x0001b2f18a38  /System/Library/PrivateFrameworks/TextInputUI.framework/TextInputUI
  0x0001b31c02a8-0x0001b31c0358  /System/Library/PrivateFrameworks/TextInputUI.framework/TextInputUI
  0x0001b1211ac8-0x0001b1212690  /System/Library/PrivateFrameworks/TextInputUI.framework/TextInputUI
  0x0001b5366000-0x0001baa0c000  /System/Library/PrivateFrameworks/TextInputUI.framework/TextInputUI
  0x0001a39f1000-0x0001a3a08000  /System/Library/PrivateFrameworks/CoreAccessories.framework/CoreAccessories
  0x0001b2c9e418-0x0001b2c9f070  /System/Library/PrivateFrameworks/CoreAccessories.framework/CoreAccessories
  0x0001b06229b0-0x0001b06282b0  /System/Library/PrivateFrameworks/CoreAccessories.framework/CoreAccessories
  0x0001b3150a98-0x0001b3150b81  /System/Library/PrivateFrameworks/CoreAccessories.framework/CoreAccessories
2017-12-12 16:18:59.529 [tid=188882][general][info] Download Manager --- RequestTask is created at download manager ---  request key is /bq/update_events-A0F8564D-26E2-45C2-A1B7-D540A2BFE249, dispatch_to_task_create delay is 0
  0x0001b5366000-0x0001baa0c000  /System/Library/PrivateFrameworks/CoreAccessories.framework/CoreAccessories
  0x0001a1361000-0x0001a1616000  /System/Library/CoreServices/RawCamera.bundle/RawCamera
  0x0001b2ad2850-0x0001b2aeb838  /System/Library/CoreServices/RawCamera.bundle/RawCamera
  0x0001b02b2360-0x0001b02f97d0  /System/Library/CoreServices/RawCamera.bundle/RawCamera
ERROR: Failed to mmap
Warning: hit breakpoint while running function, skipping commands and conditions to prevent recursion.warning: could not execute support code to read Objective-C class data in the process. This may reduce the quality of type information available.
AddressSanitizer report breakpoint hit. Use 'thread info -s' to get extended information about the report.


On Fri, Dec 8, 2017 at 10:12 AM, Kostya Serebryany <[hidden email]> wrote:
+ Devin and Kuba from Apple, who may have iOS-specific knowledge. 

We've recently made several changes that make asan consume less memory, but
* these changes are not yet in Xcode 9 (I assume)
* I am not 100% sure these changes affect iOS (we tested only on Linux, Kuba has recently applied related patches for Mac)
* Your problem might be caused by something else

Typically the biggest source of memory consumption is quarantine and the stack trace storage (stack_depot).
If you are able to set ASAN_OPTIONS env var, please try this: 
ASAN_OPTIONS=quarantine_size_mb=1:malloc_context_size=5

quarantine_size_mb=1 limits the quarantine size (and thus reduces the ability to find use-after-free)
malloc_context_size=5 reduces the length of stack traces that asan stores for future bug reporting 

If these don't help, we'll need more details (e.g. the iOS's analog of /proc/PID/maps for the process at the time of failure)

--kcc 



On Thu, Dec 7, 2017 at 12:56 PM, Michael Eisel via cfe-dev <[hidden email]> wrote:
Hi,

I'm working on an iPhone app that rarely runs out of memory normally, but with the address sanitizer on will crash after a couple minutes. I don't remember it crashing this much with Xcode 8's version of clang, but in any case I'm using Xcode 9 now. Although crashing after a couple minutes is fine for testing during development, I would like to distribute the app to internal testers with asan built into it, and I don't want to impact their experience so much. Is there any simple way to reduce memory use, e.g. by turning off some component of asan?

Thanks,
Michael

_______________________________________________
cfe-dev mailing list
[hidden email]
http://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-dev





_______________________________________________
cfe-dev mailing list
[hidden email]
http://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-dev
Reply | Threaded
Open this post in threaded view
|

Re: Address sanitizer memory usage

Eric Fiselier via cfe-dev
In reply to this post by Eric Fiselier via cfe-dev
If you're willing to do extra legwork of multiple passes maybe you could try turning off features or reducing the scope of what's instrumented.

e.g.
* only "fsanitize=address" on small parts at a time
* Turn off "detect_leaks" or "detect_stack_use_after_return" (though this one was off by default)
* try disabling "-asan-stack" or "-asan-globals"

On Tue, Dec 12, 2017 at 6:23 PM, Michael Eisel via cfe-dev <[hidden email]> wrote:
Hi, so when you say "ASAN_OPTIONS=quarantine_size_mb=1:malloc_context_size=5", do you mean that it's one key-value pair with key "ASAN_OPTIONS" and value "quarantine_size_mb=1:malloc_context_size=5" ? I tried that, but unfortunately, the phone still crashes after a few minutes of typical usage. Here's the trace:

==1020==ERROR: AddressSanitizer failed to allocate 0x80000 (524288) bytes of SetAlternateSignalStack (error code: 12)
==1020==Process memory map follows:
  0x000195805000-0x000195819000  /System/Library/PrivateFrameworks/EmojiFoundation.framework/EmojiFoundation
  0x0001b216e458-0x0001b216ee98  /System/Library/PrivateFrameworks/EmojiFoundation.framework/EmojiFoundation
  0x0001ae3f0178-0x0001ae3ff328  /System/Library/PrivateFrameworks/EmojiFoundation.framework/EmojiFoundation
  0x0001b315cf10-0x0001b315d070  /System/Library/PrivateFrameworks/EmojiFoundation.framework/EmojiFoundation
  0x0001b5366000-0x0001baa0c000  /System/Library/PrivateFrameworks/EmojiFoundation.framework/EmojiFoundation
  0x0001937e6000-0x0001937f8000  /System/Library/PrivateFrameworks/CoreFollowUp.framework/CoreFollowUp
  0x0001b20576f0-0x0001b2058280  /System/Library/PrivateFrameworks/CoreFollowUp.framework/CoreFollowUp
  0x0001add00b60-0x0001add06638  /System/Library/PrivateFrameworks/CoreFollowUp.framework/CoreFollowUp
  0x0001b31439a8-0x0001b3143c78  /System/Library/PrivateFrameworks/CoreFollowUp.framework/CoreFollowUp
  0x0001b5366000-0x0001baa0c000  /System/Library/PrivateFrameworks/CoreFollowUp.framework/CoreFollowUp
  0x00019447f000-0x00019448b000  /System/Library/PrivateFrameworks/SetupAssistantSupport.framework/SetupAssistantSupport
  0x0001b20bd288-0x0001b20be258  /System/Library/PrivateFrameworks/SetupAssistantSupport.framework/SetupAssistantSupport
  0x0001adfc1300-0x0001adfc4b38  /System/Library/PrivateFrameworks/SetupAssistantSupport.framework/SetupAssistantSupport
  0x0001b5366000-0x0001baa0c000  /System/Library/PrivateFrameworks/SetupAssistantSupport.framework/SetupAssistantSupport
  0x00019448b000-0x0001944b3000  /System/Library/PrivateFrameworks/SetupAssistant.framework/SetupAssistant
  0x0001b20be258-0x0001b20bfbc8  /System/Library/PrivateFrameworks/SetupAssistant.framework/SetupAssistant
  0x0001adfc4b38-0x0001adfcb630  /System/Library/PrivateFrameworks/SetupAssistant.framework/SetupAssistant
  0x0001b30ff0f0-0x0001b30ff5d8  /System/Library/PrivateFrameworks/SetupAssistant.framework/SetupAssistant
  0x0001b5366000-0x0001baa0c000  /System/Library/PrivateFrameworks/SetupAssistant.framework/SetupAssistant
  0x00019511e000-0x000195138000  /System/Library/PrivateFrameworks/CoreSDB.framework/CoreSDB
  0x0001b2121128-0x0001b2121540  /System/Library/PrivateFrameworks/CoreSDB.framework/CoreSDB
  0x0001ae21d1b8-0x0001ae21f770  /System/Library/PrivateFrameworks/CoreSDB.framework/CoreSDB
  0x0001b5366000-0x0001baa0c000  /System/Library/PrivateFrameworks/CoreSDB.framework/CoreSDB
  0x000195819000-0x00019592c000  /System/Library/PrivateFrameworks/IMDPersistence.framework/IMDPersistence
  0x0001b216ee98-0x0001b2170ae8  /System/Library/PrivateFrameworks/IMDPersistence.framework/IMDPersistence
2017-12-12 16:18:59.529 [tid=189434][general][info] Download Manager --- Request is dispatched to download manager --- request key is /bq/update_events-A0F8564D-26E2-45C2-A1B7-D540A2BFE249
  0x0001b3151788-0x0001b31517d8  /System/Library/PrivateFrameworks/IMDPersistence.framework/IMDPersistence
  0x0001ae3ff328-0x0001ae4215b0  /System/Library/PrivateFrameworks/IMDPersistence.framework/IMDPersistence
  0x0001b5366000-0x0001baa0c000  /System/Library/PrivateFrameworks/IMDPersistence.framework/IMDPersistence
  0x000195138000-0x00019518e000  /System/Library/PrivateFrameworks/IMSharedUtilities.framework/IMSharedUtilities
  0x0001b2121540-0x0001b21256c8  /System/Library/PrivateFrameworks/IMSharedUtilities.framework/IMSharedUtilities
  0x0001ae21f770-0x0001ae239ad8  /System/Library/PrivateFrameworks/IMSharedUtilities.framework/IMSharedUtilities
  0x0001b312bf50-0x0001b312bff0  /System/Library/PrivateFrameworks/IMSharedUtilities.framework/IMSharedUtilities
  0x0001b5366000-0x0001baa0c000  /System/Library/PrivateFrameworks/IMSharedUtilities.framework/IMSharedUtilities
  0x000196745000-0x0001968e4000  /System/Library/PrivateFrameworks/IMCore.framework/IMCore
  0x0001b21f60c8-0x0001b21ff2e1  /System/Library/PrivateFrameworks/IMCore.framework/IMCore
  0x0001ae607b68-0x0001ae648888  /System/Library/PrivateFrameworks/IMCore.framework/IMCore
  0x0001b31352b0-0x0001b3135cc0  /System/Library/PrivateFrameworks/IMCore.framework/IMCore
  0x0001b5366000-0x0001baa0c000  /System/Library/PrivateFrameworks/IMCore.framework/IMCore
  0x00019868f000-0x0001986ec000  /System/Library/PrivateFrameworks/IMAVCore.framework/IMAVCore
  0x0001b23195d0-0x0001b231bf70  /System/Library/PrivateFrameworks/IMAVCore.framework/IMAVCore
  0x0001aec98888-0x0001aeca8ae8  /System/Library/PrivateFrameworks/IMAVCore.framework/IMAVCore
  0x0001b3158e28-0x0001b3158fe0  /System/Library/PrivateFrameworks/IMAVCore.framework/IMAVCore
  0x0001b5366000-0x0001baa0c000  /System/Library/PrivateFrameworks/IMAVCore.framework/IMAVCore
  0x0001a7670000-0x0001a7676000  /System/Library/PrivateFrameworks/TextInputUI.framework/TextInputUI
  0x0001b2f184e0-0x0001b2f18a38  /System/Library/PrivateFrameworks/TextInputUI.framework/TextInputUI
  0x0001b31c02a8-0x0001b31c0358  /System/Library/PrivateFrameworks/TextInputUI.framework/TextInputUI
  0x0001b1211ac8-0x0001b1212690  /System/Library/PrivateFrameworks/TextInputUI.framework/TextInputUI
  0x0001b5366000-0x0001baa0c000  /System/Library/PrivateFrameworks/TextInputUI.framework/TextInputUI
  0x0001a39f1000-0x0001a3a08000  /System/Library/PrivateFrameworks/CoreAccessories.framework/CoreAccessories
  0x0001b2c9e418-0x0001b2c9f070  /System/Library/PrivateFrameworks/CoreAccessories.framework/CoreAccessories
  0x0001b06229b0-0x0001b06282b0  /System/Library/PrivateFrameworks/CoreAccessories.framework/CoreAccessories
  0x0001b3150a98-0x0001b3150b81  /System/Library/PrivateFrameworks/CoreAccessories.framework/CoreAccessories
2017-12-12 16:18:59.529 [tid=188882][general][info] Download Manager --- RequestTask is created at download manager ---  request key is /bq/update_events-A0F8564D-26E2-45C2-A1B7-D540A2BFE249, dispatch_to_task_create delay is 0
  0x0001b5366000-0x0001baa0c000  /System/Library/PrivateFrameworks/CoreAccessories.framework/CoreAccessories
  0x0001a1361000-0x0001a1616000  /System/Library/CoreServices/RawCamera.bundle/RawCamera
  0x0001b2ad2850-0x0001b2aeb838  /System/Library/CoreServices/RawCamera.bundle/RawCamera
  0x0001b02b2360-0x0001b02f97d0  /System/Library/CoreServices/RawCamera.bundle/RawCamera
ERROR: Failed to mmap
Warning: hit breakpoint while running function, skipping commands and conditions to prevent recursion.warning: could not execute support code to read Objective-C class data in the process. This may reduce the quality of type information available.
AddressSanitizer report breakpoint hit. Use 'thread info -s' to get extended information about the report.


On Fri, Dec 8, 2017 at 10:12 AM, Kostya Serebryany <[hidden email]> wrote:
+ Devin and Kuba from Apple, who may have iOS-specific knowledge. 

We've recently made several changes that make asan consume less memory, but
* these changes are not yet in Xcode 9 (I assume)
* I am not 100% sure these changes affect iOS (we tested only on Linux, Kuba has recently applied related patches for Mac)
* Your problem might be caused by something else

Typically the biggest source of memory consumption is quarantine and the stack trace storage (stack_depot).
If you are able to set ASAN_OPTIONS env var, please try this: 
ASAN_OPTIONS=quarantine_size_mb=1:malloc_context_size=5

quarantine_size_mb=1 limits the quarantine size (and thus reduces the ability to find use-after-free)
malloc_context_size=5 reduces the length of stack traces that asan stores for future bug reporting 

If these don't help, we'll need more details (e.g. the iOS's analog of /proc/PID/maps for the process at the time of failure)

--kcc 



On Thu, Dec 7, 2017 at 12:56 PM, Michael Eisel via cfe-dev <[hidden email]> wrote:
Hi,

I'm working on an iPhone app that rarely runs out of memory normally, but with the address sanitizer on will crash after a couple minutes. I don't remember it crashing this much with Xcode 8's version of clang, but in any case I'm using Xcode 9 now. Although crashing after a couple minutes is fine for testing during development, I would like to distribute the app to internal testers with asan built into it, and I don't want to impact their experience so much. Is there any simple way to reduce memory use, e.g. by turning off some component of asan?

Thanks,
Michael

_______________________________________________
cfe-dev mailing list
[hidden email]
http://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-dev




_______________________________________________
cfe-dev mailing list
[hidden email]
http://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-dev




--
-Brian

_______________________________________________
cfe-dev mailing list
[hidden email]
http://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-dev
Reply | Threaded
Open this post in threaded view
|

Re: Address sanitizer memory usage

Eric Fiselier via cfe-dev


On Tue, Dec 12, 2017 at 6:40 PM, Brian Cain <[hidden email]> wrote:
If you're willing to do extra legwork of multiple passes maybe you could try turning off features or reducing the scope of what's instrumented.

e.g.
* only "fsanitize=address" on small parts at a time

This will help against CPU overhead, but not against memory overhead. 
 
* Turn off "detect_leaks" or "detect_stack_use_after_return" (though this one was off by default)
* try disabling "-asan-stack" or "-asan-globals"

On Tue, Dec 12, 2017 at 6:23 PM, Michael Eisel via cfe-dev <[hidden email]> wrote:
Hi, so when you say "ASAN_OPTIONS=quarantine_size_mb=1:malloc_context_size=5", do you mean that it's one key-value pair with key "ASAN_OPTIONS" and value "quarantine_size_mb=1:malloc_context_size=5" ? I tried that, but unfortunately, the phone still crashes after a few minutes of typical usage. Here's the trace:

==1020==ERROR: AddressSanitizer failed to allocate 0x80000 (524288) bytes of SetAlternateSignalStack (error code: 12)
==1020==Process memory map follows:
  0x000195805000-0x000195819000  /System/Library/PrivateFrameworks/EmojiFoundation.framework/EmojiFoundation
  0x0001b216e458-0x0001b216ee98  /System/Library/PrivateFrameworks/EmojiFoundation.framework/EmojiFoundation
  0x0001ae3f0178-0x0001ae3ff328  /System/Library/PrivateFrameworks/EmojiFoundation.framework/EmojiFoundation
  0x0001b315cf10-0x0001b315d070  /System/Library/PrivateFrameworks/EmojiFoundation.framework/EmojiFoundation
  0x0001b5366000-0x0001baa0c000  /System/Library/PrivateFrameworks/EmojiFoundation.framework/EmojiFoundation
  0x0001937e6000-0x0001937f8000  /System/Library/PrivateFrameworks/CoreFollowUp.framework/CoreFollowUp
  0x0001b20576f0-0x0001b2058280  /System/Library/PrivateFrameworks/CoreFollowUp.framework/CoreFollowUp
  0x0001add00b60-0x0001add06638  /System/Library/PrivateFrameworks/CoreFollowUp.framework/CoreFollowUp
  0x0001b31439a8-0x0001b3143c78  /System/Library/PrivateFrameworks/CoreFollowUp.framework/CoreFollowUp
  0x0001b5366000-0x0001baa0c000  /System/Library/PrivateFrameworks/CoreFollowUp.framework/CoreFollowUp
  0x00019447f000-0x00019448b000  /System/Library/PrivateFrameworks/SetupAssistantSupport.framework/SetupAssistantSupport
  0x0001b20bd288-0x0001b20be258  /System/Library/PrivateFrameworks/SetupAssistantSupport.framework/SetupAssistantSupport
  0x0001adfc1300-0x0001adfc4b38  /System/Library/PrivateFrameworks/SetupAssistantSupport.framework/SetupAssistantSupport
  0x0001b5366000-0x0001baa0c000  /System/Library/PrivateFrameworks/SetupAssistantSupport.framework/SetupAssistantSupport
  0x00019448b000-0x0001944b3000  /System/Library/PrivateFrameworks/SetupAssistant.framework/SetupAssistant
  0x0001b20be258-0x0001b20bfbc8  /System/Library/PrivateFrameworks/SetupAssistant.framework/SetupAssistant
  0x0001adfc4b38-0x0001adfcb630  /System/Library/PrivateFrameworks/SetupAssistant.framework/SetupAssistant
  0x0001b30ff0f0-0x0001b30ff5d8  /System/Library/PrivateFrameworks/SetupAssistant.framework/SetupAssistant
  0x0001b5366000-0x0001baa0c000  /System/Library/PrivateFrameworks/SetupAssistant.framework/SetupAssistant
  0x00019511e000-0x000195138000  /System/Library/PrivateFrameworks/CoreSDB.framework/CoreSDB
  0x0001b2121128-0x0001b2121540  /System/Library/PrivateFrameworks/CoreSDB.framework/CoreSDB
  0x0001ae21d1b8-0x0001ae21f770  /System/Library/PrivateFrameworks/CoreSDB.framework/CoreSDB
  0x0001b5366000-0x0001baa0c000  /System/Library/PrivateFrameworks/CoreSDB.framework/CoreSDB
  0x000195819000-0x00019592c000  /System/Library/PrivateFrameworks/IMDPersistence.framework/IMDPersistence
  0x0001b216ee98-0x0001b2170ae8  /System/Library/PrivateFrameworks/IMDPersistence.framework/IMDPersistence
2017-12-12 16:18:59.529 [tid=189434][general][info] Download Manager --- Request is dispatched to download manager --- request key is /bq/update_events-A0F8564D-26E2-45C2-A1B7-D540A2BFE249
  0x0001b3151788-0x0001b31517d8  /System/Library/PrivateFrameworks/IMDPersistence.framework/IMDPersistence
  0x0001ae3ff328-0x0001ae4215b0  /System/Library/PrivateFrameworks/IMDPersistence.framework/IMDPersistence
  0x0001b5366000-0x0001baa0c000  /System/Library/PrivateFrameworks/IMDPersistence.framework/IMDPersistence
  0x000195138000-0x00019518e000  /System/Library/PrivateFrameworks/IMSharedUtilities.framework/IMSharedUtilities
  0x0001b2121540-0x0001b21256c8  /System/Library/PrivateFrameworks/IMSharedUtilities.framework/IMSharedUtilities
  0x0001ae21f770-0x0001ae239ad8  /System/Library/PrivateFrameworks/IMSharedUtilities.framework/IMSharedUtilities
  0x0001b312bf50-0x0001b312bff0  /System/Library/PrivateFrameworks/IMSharedUtilities.framework/IMSharedUtilities
  0x0001b5366000-0x0001baa0c000  /System/Library/PrivateFrameworks/IMSharedUtilities.framework/IMSharedUtilities
  0x000196745000-0x0001968e4000  /System/Library/PrivateFrameworks/IMCore.framework/IMCore
  0x0001b21f60c8-0x0001b21ff2e1  /System/Library/PrivateFrameworks/IMCore.framework/IMCore
  0x0001ae607b68-0x0001ae648888  /System/Library/PrivateFrameworks/IMCore.framework/IMCore
  0x0001b31352b0-0x0001b3135cc0  /System/Library/PrivateFrameworks/IMCore.framework/IMCore
  0x0001b5366000-0x0001baa0c000  /System/Library/PrivateFrameworks/IMCore.framework/IMCore
  0x00019868f000-0x0001986ec000  /System/Library/PrivateFrameworks/IMAVCore.framework/IMAVCore
  0x0001b23195d0-0x0001b231bf70  /System/Library/PrivateFrameworks/IMAVCore.framework/IMAVCore
  0x0001aec98888-0x0001aeca8ae8  /System/Library/PrivateFrameworks/IMAVCore.framework/IMAVCore
  0x0001b3158e28-0x0001b3158fe0  /System/Library/PrivateFrameworks/IMAVCore.framework/IMAVCore
  0x0001b5366000-0x0001baa0c000  /System/Library/PrivateFrameworks/IMAVCore.framework/IMAVCore
  0x0001a7670000-0x0001a7676000  /System/Library/PrivateFrameworks/TextInputUI.framework/TextInputUI
  0x0001b2f184e0-0x0001b2f18a38  /System/Library/PrivateFrameworks/TextInputUI.framework/TextInputUI
  0x0001b31c02a8-0x0001b31c0358  /System/Library/PrivateFrameworks/TextInputUI.framework/TextInputUI
  0x0001b1211ac8-0x0001b1212690  /System/Library/PrivateFrameworks/TextInputUI.framework/TextInputUI
  0x0001b5366000-0x0001baa0c000  /System/Library/PrivateFrameworks/TextInputUI.framework/TextInputUI
  0x0001a39f1000-0x0001a3a08000  /System/Library/PrivateFrameworks/CoreAccessories.framework/CoreAccessories
  0x0001b2c9e418-0x0001b2c9f070  /System/Library/PrivateFrameworks/CoreAccessories.framework/CoreAccessories
  0x0001b06229b0-0x0001b06282b0  /System/Library/PrivateFrameworks/CoreAccessories.framework/CoreAccessories
  0x0001b3150a98-0x0001b3150b81  /System/Library/PrivateFrameworks/CoreAccessories.framework/CoreAccessories
2017-12-12 16:18:59.529 [tid=188882][general][info] Download Manager --- RequestTask is created at download manager ---  request key is /bq/update_events-A0F8564D-26E2-45C2-A1B7-D540A2BFE249, dispatch_to_task_create delay is 0
  0x0001b5366000-0x0001baa0c000  /System/Library/PrivateFrameworks/CoreAccessories.framework/CoreAccessories
  0x0001a1361000-0x0001a1616000  /System/Library/CoreServices/RawCamera.bundle/RawCamera
  0x0001b2ad2850-0x0001b2aeb838  /System/Library/CoreServices/RawCamera.bundle/RawCamera
  0x0001b02b2360-0x0001b02f97d0  /System/Library/CoreServices/RawCamera.bundle/RawCamera
ERROR: Failed to mmap
Warning: hit breakpoint while running function, skipping commands and conditions to prevent recursion.warning: could not execute support code to read Objective-C class data in the process. This may reduce the quality of type information available.
AddressSanitizer report breakpoint hit. Use 'thread info -s' to get extended information about the report.


On Fri, Dec 8, 2017 at 10:12 AM, Kostya Serebryany <[hidden email]> wrote:
+ Devin and Kuba from Apple, who may have iOS-specific knowledge. 

We've recently made several changes that make asan consume less memory, but
* these changes are not yet in Xcode 9 (I assume)
* I am not 100% sure these changes affect iOS (we tested only on Linux, Kuba has recently applied related patches for Mac)
* Your problem might be caused by something else

Typically the biggest source of memory consumption is quarantine and the stack trace storage (stack_depot).
If you are able to set ASAN_OPTIONS env var, please try this: 
ASAN_OPTIONS=quarantine_size_mb=1:malloc_context_size=5

quarantine_size_mb=1 limits the quarantine size (and thus reduces the ability to find use-after-free)
malloc_context_size=5 reduces the length of stack traces that asan stores for future bug reporting 

If these don't help, we'll need more details (e.g. the iOS's analog of /proc/PID/maps for the process at the time of failure)

--kcc 



On Thu, Dec 7, 2017 at 12:56 PM, Michael Eisel via cfe-dev <[hidden email]> wrote:
Hi,

I'm working on an iPhone app that rarely runs out of memory normally, but with the address sanitizer on will crash after a couple minutes. I don't remember it crashing this much with Xcode 8's version of clang, but in any case I'm using Xcode 9 now. Although crashing after a couple minutes is fine for testing during development, I would like to distribute the app to internal testers with asan built into it, and I don't want to impact their experience so much. Is there any simple way to reduce memory use, e.g. by turning off some component of asan?

Thanks,
Michael

_______________________________________________
cfe-dev mailing list
[hidden email]
http://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-dev




_______________________________________________
cfe-dev mailing list
[hidden email]
http://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-dev




--
-Brian


_______________________________________________
cfe-dev mailing list
[hidden email]
http://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-dev