Memory leak in long-running scripts when using OAuth #1100
Comments
|
Are you saying that this only happens with OAuth and not Basic authentication? For OAuth we use the Can you install a memory debugging tool for Python to see exactly which data structures are consuming all this memory? |
|
Yes. Earlier we are using below given code and python lib. also please suggest a good memory debugging tool for Python . Python == 3.8.5 from exchangelib import DELEGATE,Account,Credentials,Configuration
credentials = Credentials(
username= 'dummyy@dummy.com',
password='******'
)
config = Configuration(server='outlook.office365.com', credentials=credentials)
account = Account(
primary_smtp_address='dummy@gdummyy.com',
credentials=credentials,
autodiscover=False,
config=config,
access_type=DELEGATE
)
mails = list(account.inbox.filter(is_read=False).order_by('datetime_received')[:1]) |
|
You can use the built-in |
|
Thanks Erik, we are looking into memory debugging tool, meanwhile After 100 % memory utilization our server crashed and getting below error: |
|
Hi Erik , I have implemented tracemalloc memory debugging tool in our application and take the snapshot dump (attaching the latest and old snapshot dump) and also I have compare both snapshot (attaching the result). Please help me out to resolve this issue |
|
In the Also, you mention memory increasing for every request, using some sort of polling every minute? The above snippet does not do that. It just fetches email data and exits. |
|
Yes Erik, but similar code is running fine when we are using basic authentication: from exchangelib import DELEGATE,Account,Credentials,Configuration
credentials = Credentials(
username= 'dummyy@dummy.com',
password='******'
)
config = Configuration(server='outlook.office365.com', credentials=credentials)
account = Account(
primary_smtp_address='dummy@gdummyy.com',
credentials=credentials,
autodiscover=False,
config=config,
access_type=DELEGATE
)
mails = list(account.inbox.filter(is_read=False).order_by('datetime_received')[:1])but After doing only below changes to implement OAuth on office 365 in our code memory usage going up with every request. Python - 3.8.5 from exchangelib import OAuth2Credentials,Build,Version,Configuration, OAUTH2,Account,IMPERSONATION
## Removing below info client _id client_secret, tenant_id because it is confidential
CLIENT_ID = '*****************'
CLIENT_SECRET = '***********'
TENANT_ID = '****'
version = Version(build=Build(15, 0, 12, 34))
credentials = OAuth2Credentials(
client_id=CLIENT_ID, client_secret=CLIENT_SECRET, tenant_id=TENANT_ID
)
config = Configuration(service_endpoint = 'https://outlook.office365.com/EWS/Exchange.asmx',
credentials=credentials,
auth_type=OAUTH2 ,
version=version)
account = Account(
primary_smtp_address='dummy@sdummy.com', ## need to put valid adress
credentials=credentials,
autodiscover=False,
config=config,
access_type=IMPERSONATION
)
mails = list(account.inbox.filter(is_read=False).only(
'is_read', 'subject', 'body','text_body','datetime_received',
'sender','to_recipients','cc_recipients','bcc_recipients',
'attachments','importance'
).order_by('datetime_received')[:1]) |
|
Ok, so you're changing both the authentication mechanism and the exchangelib version. Please try the same, but with the same exchangelib version both places. It's not unexpected that memory will increase while the last line is running. exchangelib fetches items in chunks when there are many items. In the above, you're fetching full items, including attachments, so memory usage may be considerable, depending on the number of items and the attachment sizes. |
|
Hi Erik , we have updated the exchange lib api from 3.1.1 to 4.7.4 with old code (given below) on our server 2 days back and its working fine , below is the code which is on my server on which we only updated exchange lib api from 3.1.1 to 4.7.4 . from exchangelib import DELEGATE,Account,Credentials,Configuration
credentials = Credentials(
username= 'dummyy@dummy.com',
password='******'
)
config = Configuration(server='outlook.office365.com', credentials=credentials)
account = Account(
primary_smtp_address='dummy@gdummyy.com',
credentials=credentials,
autodiscover=False,
config=config,
access_type=DELEGATE
)
mails = list(account.inbox.filter(is_read=False).only( 'is_read', 'subject', 'body','text_body','datetime_received', 'sender','to_recipients','cc_recipients','bcc_recipients','attachments','importance').order_by('datetime_received')[:1])*** So my issue is as you aware that Office 365 is deprecating Basic authentication, so i need to implement OAuth on office 365 and when we implement OAuth on office 365 in our code we are facing memory issue (memory usage going up with every request), only because of these below changes which we coded for OAuth on office 365. I am also attaching some more analysis and snap shot dump from trace malloc. Code which we changed to implement for OAuth on office 365 from exchangelib import OAuth2Credentials,Build,Version,Configuration, OAUTH2,Account,IMPERSONATION
## Removing below info client _id client_secret, tenant_id because it is confidential
CLIENT_ID = '*****************'
CLIENT_SECRET = '***********'
TENANT_ID = '****'
version = Version(build=Build(15, 0, 12, 34))
credentials = OAuth2Credentials(
client_id=CLIENT_ID, client_secret=CLIENT_SECRET, tenant_id=TENANT_ID
)
config = Configuration(service_endpoint = 'https://outlook.office365.com/EWS/Exchange.asmx',
credentials=credentials,
auth_type=OAUTH2 ,
version=version)
account = Account(
primary_smtp_address='dummy@sdummy.com', ## need to put valid adress
credentials=credentials,
autodiscover=False,
config=config,
access_type=IMPERSONATION
)
mails = list(account.inbox.filter(is_read=False).only(
'is_read', 'subject', 'body','text_body','datetime_received',
'sender','to_recipients','cc_recipients','bcc_recipients',
'attachments','importance'
).order_by('datetime_received')[:1]) |
|
This could be due to a memory leak in the from exchangelib.protocol import Protocol
Protocol.MAX_SESSION_USAGE_COUNT = 10 # Or some other low number |
|
implemented give above code as you suggested but no improvement found still facing same issue.. |
|
Possible fix posted in #1090 (comment) |
|
still facing same issue after implementing #1090 , |
ecederstrand
changed the title
|
Facing the same RAM memory increasing issue, on a long-running script that connects every 5 seconds to an outlook mailbox using OAuth2Credentials, Configuration and Account from exchangelib: oauth2_credentials = json.loads(item['oauth2_credentials'])
credentials = OAuth2Credentials(
client_id=oauth2_credentials['client_id'],
client_secret=oauth2_credentials['client_secret'],
tenant_id=oauth2_credentials['tenant_id'],
identity=Identity(primary_smtp_address=item['login_email'])
)
config = Configuration(
server=item['inbound_email_server'],
credentials=credentials,
auth_type=OAUTH2,
)
account = Account(
primary_smtp_address=item['login_email'],
config=config,
autodiscover=False,
access_type=DELEGATE,
) |
|
Is this issue fixed? The deadline for basic authentication is 12/31/2022. We also need the fix to use OAuth. |
|
Nope, to my knowledge it's not fixed. I also cannot reproduce this in my local setup. You'll need to start debugging in your own processes where the memory usage is piling up. As a workaround, you can have the Python process exit and restart every once in a while. |
|
Also having this exact issue with the following code: def _connect(self):
if self.exchange_oauth_tenant_id and self.exchange_oauth_client_id and self.exchange_oauth_client_secret:
credentials = OAuth2Credentials(
client_id=self.exchange_oauth_client_id,
client_secret=self.exchange_oauth_client_secret,
tenant_id=self.exchange_oauth_tenant_id,
identity=Identity(primary_smtp_address=self.email_address),
)
elif self.exchange_user and self.exchange_password:
credentials = Credentials(username=self.exchange_user, password=self.exchange_password)
else:
raise RuntimeError("No Microsoft Exchange details available")
configuration = Configuration(server=self.exchange_server, credentials=credentials)
self.exchange_account = Account(
primary_smtp_address=self.email_address, autodiscover=False, access_type=DELEGATE, config=configuration
)
return self.exchange_accountNo memory issues when username and password verification is used, but linear increase in memory usage when OAuth is used. Issue resolution:I think we have been able to solve this issue at least in our instance, similar to this solution:
except we close out the connection after every mailbox scan: @contextmanager
def connection(self) -> Iterator["ExchangeEmailSystem"]:
try:
# Open mailbox connection
self._connect()
yield self
finally:
# Close mailbox connection
self.exchange_account.protocol.close()In our code responsible for scanning the mailboxes, the with account.connection():
folder_object = [x for x in account.root.walk() if x.name == "Inbox"]
# etc.This happens routinely for all mailboxes. Maybe some increase in CPU usage but nothing that we've noticed. |
Describe the bug
connecting to the EWS with python using Exchanhelib 4.7.4 to implement OAuth on Office 365 memory usage going up with every request.
To Reproduce
Additional context
Python - 3.8.5
Exchangelib - 4.7.4
Note : hitting EWS server every minute to get the mails , earlier we are using Exchangelib Basic authentication which is working fine but now deprecating , so we just try to move on OAuth on Office 365 using exchnagelib 4.7.4 causing this issue memory usage going up with every request on our every env Dev/QA and UAT,
The text was updated successfully, but these errors were encountered: