c# - Improving azure blob storage querying speed -
we have blob storage thousands of files under same azure container. our file naming convention this:
storagename\team\subteam\filename
i'm writing tool displays files each particular subteam. code gets list of blobs container , each of tries match correct team\subteam (see below sample code).
this works extremely slow (because need go through files see if match particular subteam). there way improve speed of query? can think of optimizations such "find first file matches team looking , keep track when find different team quit early" assume bloblist sorted , wouldn't fix worst case scenario.
unfortunately splitting files under different containers not option @ time.
here sample code:
ienumerable<ilistblobitem> blobs = blobcontainer.listblobs( new blobrequestoptions() { useflatbloblisting = true, bloblistingdetails = bloblistingdetails.metadata }).oftype<cloudblob>(); foreach (var blob in blobs) { var cloudy = blob cloudblob; string blobteamid = cloudy.uri.segments[2].trim('/'); if (blobteamid != teamid) continue; //do interesting file
1st solution rest interface can pass in
http://somwhere.com/mycontainername/?restype=container&comp=list&delimiter=/&prefix=\team\subteam and return xml doc files in sub team "folder" (i know not folder looks 1 in tools)
you might need generate shared access signature able access have tag on end of url.
where shows can filter blobname prefix.
2nd solution closer want. if can use new storage client updated in azure sdk 1.3 can use
ienumerable bloblist = client.listblobswithprefix("team/subteam");
where client instance of cloudblobclient.
edit - 18 nov 2013 looks resttype no longer supported parameter , should restype. seems have happened quietly on weekend. have changed url example above.
Comments
Post a Comment