http://m.getsatisfaction.com/topics/7763639 S3 upload option to "delete all" records in an MDA table before inserting new data records 2018-08-30T18:47:49Z 2018-07-13T13:35:56Z 7763639 http://m.getsatisfaction.com/topics/7763639 7763639 S3 upload option to "delete all" records in an MDA table before inserting new data records 2018-08-30T18:47:49Z 2018-07-13T13:35:56Z Jeff Kirkpatrick http://m.getsatisfaction.com/people/7657820 https://d2r1vs3d9006ap.cloudfront.net/public/uploaded_images/11032177/860950_medium.jpg jeff_kirkpatrick We've got an issue that probably most customers have or will run into at some point.&nbsp;&nbsp;<br /><br />When we get S3 uploads and it's updated or new data records, S3 insert/upsert works just fine.<br /><br />But if there is an absence of data that was previously uploaded to S3, there's no way to clear those records.<br /><br />Would be incredibly helpful to have a "delete all" option to delete all the records in an MDA table before S3 upload job processes so that we can insert the entire fresh export of data.<br /><br />Has anyone else found a way to get around this issue? idea cross_team_dependency cross_team_dependency, shout_out_worthy 5 7 5 comment http://m.getsatisfaction.com/topics/7763639/replies/19642127 http://m.getsatisfaction.com/topics/7763639 19642127 Sagan Sherlin responded to "S3 upload option to "delete all" records in an MDA table before inserting new data records" 2018-07-16T16:34:14Z 2018-07-16T16:34:14Z Sagan Sherlin http://m.getsatisfaction.com/people/9511694 https://d2r1vs3d9006ap.cloudfront.net/public/uploaded_images/11221250/264DeathbyCat5-m_medium.jpg sagan_sherlin +1 This would be awesome!&nbsp; Right now it's a two step process...and if your forget to delete all the data at first.&nbsp; Well. More work!&nbsp;&nbsp; 0 http://m.getsatisfaction.com/topics/7763639/replies/19642163 http://m.getsatisfaction.com/topics/7763639 19642163 Mark Gecsey responded to "S3 upload option to "delete all" records in an MDA table before inserting new data records" 2018-07-16T16:48:55Z 2018-07-16T16:48:55Z Mark Gecsey http://m.getsatisfaction.com/people/9346107 https://d2r1vs3d9006ap.cloudfront.net/public/uploaded_images/11164139/photo_medium.jpg mark_gecsey +1 for us as well.&nbsp; 0 http://m.getsatisfaction.com/topics/7763639/replies/19645585 http://m.getsatisfaction.com/topics/7763639 19645585 Dan Ahrens responded to "S3 upload option to "delete all" records in an MDA table before inserting new data records" 2018-07-18T03:18:08Z 2018-07-18T03:18:08Z Dan Ahrens http://m.getsatisfaction.com/people/8169071 https://d2r1vs3d9006ap.cloudfront.net/public/uploaded_images/10797429/dan-ahrens_medium.jpg dan_ahrens Hi folks,&nbsp;<br />I wanted to share here an idea that Jeff and I had a chance to discuss via the phone. Since invariably once you drop a table, a few weeks later someone will want historical data, one way to 'get the best of both worlds' would be this approach.&nbsp;<br /><br /><b>Assumptions:</b><br />* You get a daily (or weekly or other regular cadence. For this example we'll use daily.) update file that is the entire data set and is not just incremental updates.<br />* The data file you are working with does not identify which records are to be deleted - they are just not present on the first cycle where that record is no longer valid<br /><br /><b>Approach:</b><br />* Add two columns to your MDA file: "Date last updated" (date type field) and "Current" (boolean)<br />* Create a rule that runs daily&nbsp;<i>before&nbsp;</i>your daily import that sets the value of "Current" for every record in that MDA object to null<br />* When you import your daily file, upsert your new data to the MDA object and also write the rule date to the "Date last updated" field and set the "Current" field to not null.<br /><br /><b>This will accomplish two things:</b><br />* You'll immediately be able to filter any current records by using that boolean "Current" field<br />* You'll have a historical record of the last time any individual records were updated 1 http://m.getsatisfaction.com/topics/7763639/replies/19715956 http://m.getsatisfaction.com/topics/7763639/replies/19645585 19715956 Samantha Braastad responded to "S3 upload option to "delete all" records in an MDA table before inserting new data records" 2018-08-24T13:38:59Z 2018-08-24T13:38:59Z Samantha Braastad http://m.getsatisfaction.com/people/8189521 https://d2r1vs3d9006ap.cloudfront.net/public/uploaded_images/10894897/F_medium.jpg samantha_braastad As an alternative, you can create a rule that sets the "Current" boolean to null if the "Date Last Updated" field is more then X amount of days in the past. This reduces the number of records that need to be updated each day :) 3 http://m.getsatisfaction.com/topics/7763639/replies/19728213 http://m.getsatisfaction.com/topics/7763639/replies/19645585 19728213 Jeff Kirkpatrick responded to "S3 upload option to "delete all" records in an MDA table before inserting new data records" 2018-08-30T18:47:49Z 2018-08-30T18:47:49Z Jeff Kirkpatrick http://m.getsatisfaction.com/people/7657820 https://d2r1vs3d9006ap.cloudfront.net/public/uploaded_images/11032177/860950_medium.jpg jeff_kirkpatrick Hey Samantha I just noticed this.&nbsp; nice suggestion!&nbsp; I'll try to remember for next time. 0