Intermittent Errors in Salesforce Marketing Cloud Integration #85
Description
Hi Community,
We are using the SFMC integration to sync data from SFMC to BigQuery. There are intermittent errors happening quite often. Sometimes it last for over a day, sometimes last for a couple of hours. And they can recover at sometime later on w/o any work from our side. Could you help us to understand
- the root cause of such error
- does the integration need to end up omitting some data in order to recover such error ? (or what's changed for the sync to move on w/o errors)
Thanks!
One error log example attached below
`
2021-05-27 11:55:57,858Z target - INFO replicated 6805 records from "data_extension.PROD_Businesses" endpoint
2021-05-27 11:55:58,008Z tap - INFO Getting more results from 'DataExtensionObject' endpoint
2021-05-27 11:55:58,555Z tap - ERROR <suds.sax.document.Document object at 0x7f6e142ad470>
2021-05-27 11:55:58,555Z tap - ERROR :2:62: syntax error
2021-05-27 11:55:58,555Z tap - Traceback (most recent call last):
2021-05-27 11:55:58,555Z tap - File "/root/.pyenv/versions/3.5.2/lib/python3.5/xml/sax/expatreader.py", line 210, in feed
2021-05-27 11:55:58,555Z tap - self._parser.Parse(data, isFinal)
2021-05-27 11:55:58,555Z tap - xml.parsers.expat.ExpatError: syntax error: line 2, column 62
2021-05-27 11:55:58,555Z tap -
2021-05-27 11:55:58,555Z tap - During handling of the above exception, another exception occurred:
2021-05-27 11:55:58,555Z tap -
2021-05-27 11:55:58,556Z tap - Traceback (most recent call last):
2021-05-27 11:55:58,556Z tap - File "/code/orchestrator/tap-env/lib/python3.5/site-packages/tap_exacttarget/init.py", line 136, in do_sync
2021-05-27 11:55:58,556Z tap - stream_accessor.sync()
2021-05-27 11:55:58,556Z tap - File "/code/orchestrator/tap-env/lib/python3.5/site-packages/tap_exacttarget/dao.py", line 74, in sync
2021-05-27 11:55:58,556Z tap - return self.sync_data()
2021-05-27 11:55:58,556Z tap - File "/code/orchestrator/tap-env/lib/python3.5/site-packages/tap_exacttarget/endpoints/data_extensions.py", line 283, in sync_data
2021-05-27 11:55:58,556Z tap - replication_key=replication_key)
2021-05-27 11:55:58,556Z tap - File "/code/orchestrator/tap-env/lib/python3.5/site-packages/tap_exacttarget/endpoints/data_extensions.py", line 209, in _replicate
2021-05-27 11:55:58,556Z tap - for row in result:
2021-05-27 11:55:58,556Z tap - File "/code/orchestrator/tap-env/lib/python3.5/site-packages/tap_exacttarget/client.py", line 153, in request_from_cursor
2021-05-27 11:55:58,556Z tap - response = tap_exacttarget__getMoreResults(cursor, batch_size=batch_size)
2021-05-27 11:55:58,556Z tap - File "/code/orchestrator/tap-env/lib/python3.5/site-packages/tap_exacttarget/fuel_overrides.py", line 32, in tap_exacttarget__getMoreResults
2021-05-27 11:55:58,556Z tap - obj = TapExacttarget__ET_Continue(cursor.auth_stub, cursor.last_request_id, batch_size)
2021-05-27 11:55:58,556Z target - INFO Serializing batch with 1518 messages for table data_extension.PROD_Businesses
2021-05-27 11:55:58,557Z tap - File "/code/orchestrator/tap-env/lib/python3.5/site-packages/tap_exacttarget/fuel_overrides.py", line 26, in init
2021-05-27 11:55:58,557Z tap - response = auth_stub.soap_client.service.Retrieve(ws_continueRequest)
2021-05-27 11:55:58,557Z tap - File "/code/orchestrator/tap-env/lib/python3.5/site-packages/suds/client.py", line 521, in call
2021-05-27 11:55:58,557Z tap - return client.invoke(args, kwargs)
2021-05-27 11:55:58,557Z tap - File "/code/orchestrator/tap-env/lib/python3.5/site-packages/suds/client.py", line 581, in invoke
2021-05-27 11:55:58,557Z tap - result = self.send(soapenv)
2021-05-27 11:55:58,557Z tap - File "/code/orchestrator/tap-env/lib/python3.5/site-packages/suds/client.py", line 621, in send
2021-05-27 11:55:58,557Z tap - original_soapenv=original_soapenv)
2021-05-27 11:55:58,557Z tap - File "/code/orchestrator/tap-env/lib/python3.5/site-packages/suds/client.py", line 661, in process_reply
2021-05-27 11:55:58,557Z tap - replyroot = _parse(reply)
2021-05-27 11:55:58,557Z tap - File "/code/orchestrator/tap-env/lib/python3.5/site-packages/suds/client.py", line 832, in _parse
2021-05-27 11:55:58,557Z tap - return Parser().parse(string=string)
2021-05-27 11:55:58,558Z tap - File "/code/orchestrator/tap-env/lib/python3.5/site-packages/suds/sax/parser.py", line 133, in parse
2021-05-27 11:55:58,558Z tap - sax.parse(source)
2021-05-27 11:55:58,558Z tap - File "/root/.pyenv/versions/3.5.2/lib/python3.5/xml/sax/expatreader.py", line 110, in parse
2021-05-27 11:55:58,558Z tap - xmlreader.IncrementalParser.parse(self, source)
2021-05-27 11:55:58,558Z tap - File "/root/.pyenv/versions/3.5.2/lib/python3.5/xml/sax/xmlreader.py", line 125, in parse
2021-05-27 11:55:58,558Z tap - self.feed(buffer)
2021-05-27 11:55:58,558Z tap - File "/root/.pyenv/versions/3.5.2/lib/python3.5/xml/sax/expatreader.py", line 214, in feed
2021-05-27 11:55:58,558Z tap - self._err_handler.fatalError(exc)
2021-05-27 11:55:58,558Z tap - File "/root/.pyenv/versions/3.5.2/lib/python3.5/xml/sax/handler.py", line 38, in fatalError
2021-05-27 11:55:58,558Z tap - raise exception
2021-05-27 11:55:58,558Z tap - xml.sax._exceptions.SAXParseException: :2:62: syntax error
2021-05-27 11:55:58,558Z tap - ERROR Failed to sync endpoint, moving on!`