The Download Manager MUST satisfy the basic download needs of OTA Updates.
YES done in 1.0
DLMGR_A_2
The Download Manager MUST satisfy the basic download needs of Market.
YES done in 1.0
DLMGR_A_3
The Download Manager MUST satisfy the basic download needs of Gmail.
YES done in 1.0
DLMGR_A_4
The Download Manager MUST satisfy the basic download needs of the Browser.
YES done in 1.0
DLMGR_A_5
Downloads MUST happen and continue in the background, independently of the application in front.
YES done in 1.0
DLMGR_A_6
Downloads MUST be reliable even on unreliable network connections, within the capabilities of the underlying protocol and of the server.
YESne in 1.0
DLMGR_A_7
User-accessible files MUST be stored on the external storage card, and only files that are explicitly supported by an installed application must be downloaded.
/td>
YES done in 1.0
DLMGR_A_8
OMA downloads SHOULD be supported.
NO deferred beyond Cupcake (not enough time)
DLMGR_A_9
The Download Manager SHOULD only download when using high-speed (3G/wifi) links, or only when not roaming (if that can be detected).
NO erred beyond Cupcake (not enough time)
DLMGR_A_10
Downloads SHOULD be user-controllable (if allowed by the initiating application), including pause, resume, and restart of failed downloads.
4001
NO
Requirements for Cupcake
Req#
Description
Detailed Requirements
Status
DLMGR_C_1
The Download Manager SHOULD resume downloads where the socket got cleanly closed while the download was incomplete (common behavior on proxies and Google servers)
5
YES done in Cupcake
DLMGR_C_2
The Download Manager SHOULD retry/resume downloads instead of aborting when the server returns a HTTP code 503
2006
YES done in Cupcake
DLMGR_C_3
The Download Manager SHOULD randomize the retry delay
2007
YES done in Cupcake
DLMGR_C_4
The Download Manager SHOULD use the retry-after header (delta-seconds) for 503 responses
2008
YES done in Cupcake
DLMGR_C_5
The Download Manager MAY hide columns that aren't strictly necessary
1010
YES done in Cupcake
DLMGR_C_6
The Download Manager SHOULD allow the initiating app to pause an ongoing download
1011
YES done in Cupcake
DLMGR_C_7
The Download Manager SHOULD not display multiple notification icons for completed downloads.
4002
NO deferred beyond Cupcake (no precise cifications, need framework support)
DLMGR_C_8
The Download Manager SHOULD delete old files from /cache
3006
NO deferred beyond Cupcake (no precise specifications)
DLMGR_C_9
The Download Manager SHOULD handle redirects
2009
YES done in Cupcake
Requirements for Donut
Req#
Description
Detailed Requirements
Status
Technology Evaluation
ALSO See also future directions for possible additional technical changes.
Possible scope for Cupcake
Because there is no testing environment in place in the 1.0 code, and because the schedule between
1.0 and Cupcake doesn't leave enough time to develop a testing environment solid enough to be able
to test the database upgrades from 1.0 database scheme to a new scheme, any work done in Cupcake will
have to work within the existing 1.0 database scheme.
Reducing the number of classes
Each class in the system has a measurable RAM cost (because of the associated Class objects),
and therefore reducing the number of classes when possible or relevant can reduce the memory
requirements. That being said, classes that extend system classes and are necessary for the
operation of the download manager can't be removed.
Class
Comments
com.android.providers.downloads.Constants
Only contains constants, can be merged into another class.
com.android.providers.downloads.DownloadFileInfo
Should be merged with DownloadInfo.
com.android.providers.downloads.DownloadInfo
Once we only store information in RAM about downloads that are explicitly active, can be merged with DownloadThread/td>
Can probably be made to implement Runnable instead and merged into another class, can be minated if the download manager can be re-architected to not depend on ContentObserver any more.
com.android.providers.downloads.DownloadThread
Can probably be made to implement Runnable instead. Unclear whether this can be eliminated as we will probably d one object that represents an ongoing download (unless the entire state can be stored on the stack with primitive types, which is unlikely).
com.android.providers.downloads.Helpers
Can't be instantiated, can be merged into another class.
com.android.providers.downloads.Helpers.Lexer
Keeps state about an ongoing lex, can probably be merged into another class by making the lexer synchronized, ce the operation is short-lived.
Reducing the list of visible columns
Security in the download provider is primarily enforced with two separate mechanisms:
Column restrictions, such that only a small number of the download provider's columns can be read or queried by applications.
UID restrictions, such that only the application that initiated a download can access information about that download.
The first mechanism is expected to be fairly robust (the implementation is quite simple, based on projection maps, which are highly
structured), but the second one relies on arbitrary strings (URIs and SQL fragments) passed by applications and is therefore at a
higher risk of being compromised. Therefore, sensitive information stored in unrestricted columns (for which the first mechanism
doesn't apply) is at a greater risk than other information.
Here's the list of columns that can currently be read/queried, with comments:
Column
Notes
_ID
Needs to be visible so that the app can uniquely identify downloads. No security concern: those numbers are sequential and aren't hard to guess.
_DATA
Probably should not be visible to applications. WARNING Security concern: This holds filenames, including those of private files. While file permissions are posed to kick in and protect the files, hiding private filenames deeper in would probably be a reasonable idea.
MIMETYPE
Needs to be visible so that app can display the icon matching the mime type. Intended to be visible by 3rd-party download UIs. TODO Security TBD before we lement support for 3rd-party UIs.
VISIBILITY
Needs to be visible in case an app has both visible and invisible downloads. No obvious security concern.
DESTINATION
Needs to be visible in case an app has multiple destinations and wants to distinguish between them. Also used internally by the download manager. No obvious urity concern.
STATUS
Needs to be visible (1004). No obvious security concern.
LAST_MODIFICATION
Needs to be visible, e.g. so that apps can sort downloads by date of last activity, or discard old downloads. No obvious security concern.
NOTIFICATION_PACKAGE
Allows individual apps running under shared UIDs to identify their own downloads. No security concern: can be queried through package manager.
NOTIFICATION_CLASS
See NOTIFICATION_PACKAGE.
TOTAL_BYTES
Needs to be visible so that the app can display a progress bar. No obvious security concern. Intended to be visible by 3rd-party download UIs.
CURRENT_BYTES
See TOTAL_BYTES.
TITLE
Intended to be visible by 3rd-party download UIs. TODO Security and Privacy TBD before we implement support for 3rd-party UIs.
DESCRIPTION
See TITLE.
Hiding the URI column
The URI column is visible to the initiating application, which is a mild security risk. It should be hidden, but the OTA update mechanism relies on it to check duplicate downloads and to play the download that's currently ongoing in the settings app. If another string column was exposed to the initiating applications, the OTA update mechanism could use that one, and URI could n be hidden. For Cupcake, without changing the database schema, the ENTITY column could be re-used as it's currently unused.
Handling redirects
There are two important aspects to handle redirects:
Storing the intermediate URIs in the provider.
Protecting against redirect loops.
If the URI column gets hidden, it could be used to store the intermediate URIs. After 1.0 the only available integer columns were METHOD and CONTROL. CONTROL was re-exposed to applications and can't be used. METHOD is slated to be re-used for 503 retry-after delays. It could be split into two halves, one for retry-after and one for the redirect nt. It would make more sense to count the redirect loop with FAILED_CONNECTIONS, but since there's already quite some code using it it'd take a bit more effort. Ideally handling of redirects ld be delayed until a future release, with a cleanup of the database schema (going along with the cleanup of the handling of filenames).
Because of the pattern used to read/write DownloadInfo and DownloadProvider, it's impractical to store multiple small integers into a large one. Therefore, since there are no eger columns left in the database, redirects will have to wait beyond Cupcake.
ContentProvider for download UI
In order to allow a UI that can "see" all the relevant downloads, there'll need to be a separate URI (or set of URIs) in the content provider: trying to use the exact same URIs for regular download control for UI purposes (distinguishing them based on the permissions of the caller) will break down if a same app (or actually a same UID) tries to do both. It'll also break down if the system process tries to do ular download activities, since it has all permissions.
Beyond that, there's little technical challenge: there are already mechanisms in place to restrict the list of columns that can be inserted, queried and updated (inserting of course makes no sense through UI channel), they just need to be duplicated for the case of the UI. The download provider also knows how to check the permissions of its caller, there isn't anything new here.
Getting rid of OTHER_UID
Right now OTHER_UID is used by checkin/update to allow the settings app to display the name of an ongoing OTA update, and by Market to allow the system to install the new apks. It is however a gerous feature, at least because it touches a part of the code that is critical to the download manager security (separation of applications).
Getting rid of OTHER_UID would be beneficial for the download manager, but the existing functionality has to be worked around. At this point, the idea that I consider the most likely would be have checkin and market implement =ContentProvider= wrappers around their downloads, and expose those content providers to whichever app they want, with whichever security mechanism they wish to have.
Only using SDK APIs.
It'd be good if the download manager could be built against the SDK as much as possible.
Here's the list of APIs as of Nov 5 2008 that aren't in the current public API but that are used by the download manager:
<> Codes 3xx weren't considered relevant when those requirements were written
Req#
History
Status
Feature
Description
Notes
1xxx
N/A
API
1001
1.0 YES
Download Manager API
The download manager provides an API that allows applications to initiate downloads.
1002
1.0 YES
Cookies
The download manager API allows applications to pass cookies to the download manager.
1003
1.0 YES
Security
The download manager provides a mechanism that prevents arbitrary applications from accessing meta-data about rent and past downloads.
In 1.0, known holes between apps
1004
1.0 YES
Status to Initiator
The download manager allows the application initiating a download to query the status of that download.
1005
1.0 YES
Cancel by Initiator
The download manager allows the application initiating a download to cancel that download.
&p;
1006
1.0 YES
Notify Initiator
The download manager notifies the application initiating a download when the download completes (with cess or failure)
1007
1.0 YES
Fire and Forget
The download manager does not rely on the initiating application when saving the file to a shared filesystem a
1008
1.0 YES
Short User-Readable Description
The download manager allows the initiating application to optionally provide a user-readable rt description of the download
1009
1.0 YES
Long User-Readable Description
The download manager allows the initiating application to optionally provide a user-readable l description of the download
1010
Cupcake YES
Cupcake P3 YES
Restrict column access
The download provider doesn't allow access to columns that aren't strictly essary.
1011
Cupcake YES
Cupcake P2 YES
Pause downloads
The download provider allows the application initiating a download to pause that download.td>
2xxx
N/A
HTTP
2001
1.0 YES
HTTP Support
The download manager supports all the features of HTTP 1.1 (RFC 2616) that are relevant to file downloads.
2002
1.0 YES
Resume Downloads
The download manager resumes downloads that get interrupted.
2003
1.0 YES
Flaky Downloads
The download manager has mechanism that prevent excessive retries of downloads that consistently fail or get errupted.
2004
1.0 YES
Resume after Reboot
The download manager resumes downloads across device reboots.
2005
Cupcake YES
Cupcake P2 YES
Resume after socket closed
The download manager resumes incomplete downloads after the socket gets anly closed
This is necessary in order to reliably download through GFEs, though it pushes us further away from being able to download from servers that don't implement pipelining.
2006
Cupcake YES
Cupcake P2 YES
Resume after 503
The download manager resumes or retries downloads when the server returns an HTTP code
2007
Cupcake YES
Cupcake P2 YES
Random retry delay
The download manager uses partial randomness when retrying downloads on an exponential koff pattern.
2008
Cupcake YES
Cupcake P2 YES
Retry-after delta-seconds
The download manager uses the retry-after header in a 503 response to decide n to retry the request
Handling of absolute dates will be covered by a separate requirement.
2009
Cupcake YES
Cupcake P2 YES
Redirects
The download manager handles common redirects.
25xx
N/A
HTTP/Conditions
3xxx
N/A
Storage
3001
1.0 YES
File Storage
The download manager stores the results of downloads in persistent storage.
3002
1.0 YES
Max File Size
The download manager is able to handle large files (order of magnitude: 50MB)
3003
1.0 YES
Destination File Permissions
The download manager restricts access to the internal storage to applications with the ropriate permissions
3004
1.0 YES
Initiator File Access
The download manager allows the initiating application to access the destination file
d>
3005
1.0 YES
File Types
The download manager does not save files that can't be displayed by any currently installed application
sp;
3006
Cupcake NO
Cupcake P3 NO
Old Files in /cache
The download manager deletes old files in /cache
35xx
N/A
Storage/Filename
4xxx
N/A
UI
4001
1.0 NO
Download Manager UI
The download manager provides a UI that lets user get information about current downloads and control them.td>
Didn't get spec on time to be able to even consider it.
4002
Cupcake NO
Cupcake P2 NO
Single Notification Icon
The download manager displays a single icon in the notification bar regardless of number of ongoing and completed downloads.
No spec in Cupcake timeframe.
5xxx
N/A
MIME
52xx
N/A
MIME/DRM
5201
1.0 YES
DRM
The download manager respects the DRM information it receives with the responses
[reverse dependency] GMail depends on the download manager to download attachments.
[reverse dependency] OTA Update depends on the download manager to download system images.
[reverse dependency] Vending Machine depends on the download manager to download content.
[reverse dependency] Browser depends on the download manager to download non-browsable.
Download Manager depends on a notification system to let the user know when downloads complete or otherwise require attention.
Download Manager depends on a mechanism to share files with another app (letting apps access its files, or accessing apps' files).
Download Manager depends on the ability to associate individual downloads with separate applications and to restrict apps to only access their own downloads.
Download Manager depends on an HTTP stack that predictably processes all relevant kinds of HTTP requests and responses.
Download Manager depends on a connectivity manager that reports accurate information about the status of the different data connections.
Interface Documentation
WARNING Since none of those APIs are public, they are all subject to change. If you're working in the Android source tree, do NOT use the explicit values, ONLY use the symbolic stants, unless you REALLY know what you're doing and are willing to deal with the consequences; you've been warned.
The various constants that are meant to be used by applications are all defined in the android.provider.Downloads class. Whenever possible, the constants should be used instead of the explicit ues.
Permissions
Constant name
Permission name
Access restrictions
Description
Downloads.PERMISSION_ACCESS
"android.permission.ACCESS_DOWNLOAD_MANAGER"
Signature or System
Applications that want to access the nload Manager MUST have this permission.
The download manager holds this mission, and the receivers through which applications get intents of completed downloads SHOULD require this permission from the sender
Content Provider
The primary interface that applications use to communicate with the download manager is exposed as a ContentProvider.
URIs
The URIs for the Content Provider are:
Constant name
URI
Description
Downloads.CONTENT_URI
Uri.parse("content://downloads/download")
The URI of the whole Content Provider, used to insert new rows, or to query all rows.td>
N/A
ContentUris.withAppendedId(CONTENT_URI, <id>)
The URI of an individual download. <id> is the value of the Download._ID umn
Columns
The following columns are available:
Constant name
Column name
SQL Type
Access
Description
Notes
Downloads._ID
"_id"
Integer
Read
Inherited from BaseColumns._ID.
Downloads.URI
"uri"
Text
Init
Used to be readable.
Downloads.APP_DATA
"entity"
Text
Init/Read/Modify
Actual column name will change in the future.
Downloads.NO_INTEGRITY
"no_integrity"
Boolean
Init
Used to be readable.
Downloads.FILENAME_HINT
"hint"
Text
Init
Used to be readable.
Downloads._DATA
"_data"
Text
Read
Used to be Downloads.FILENAME and "filename".
Downloads.MIMETYPE
"mimetype"
Text
Init/Read
Downloads.DESTINATION
"destination"
Integer
Init
See Destination codes for ails of legal values. Used to be readable.
Requires the android.permission.ESS_DOWNLOAD_MANAGER_ADVANCED permission. Used to be readable and writable. Might disappear entirely if possible.
Downloads.TITLE
"title"
String
Init/Read/Modify
Downloads.DESCRIPTION
"description"
String
Init/Read/Modify
Destination Values
Constants name
Constant value
Description
Notes
Downloads.DESTINATION_EXTERNAL
0
Saves the file to the SD card.
Default value. Fails is SD card is not present.
Downloads.DESTINATION_CACHE_PARTITION
1
Saves the file to the internal cache partition.
Requires the "android.permission.ESS_DOWNLOAD_MANAGER_ADVANCED" permission
Downloads.DESTINATION_CACHE_PARTITION_PURGEABLE
2
Saves the file to the internal cache partition.
The download can get deleted at any e by the download manager when it needs space.
Visibility Values
Constants name
Constant value
Description
Notes
Downloads.VISIBILITY_VISIBLE_NOTIFY_COMPLETED
0
The download is visible in download UIs, and it shows up in the notification area during download after completion.
Default value for external downloads.
Downloads.VISIBILITY_VISIBLE
1
The download is visible in download UIs, and it shows up in the notification area during download but not after pletion.
Downloads.VISIBILITY_HIDDEN
2
The download is hidden from download UIs and doesn't show up in the notification area.
Default value for ernal downloads.
Control Values
Constants name
Constant value
Description
Notes
Downloads.CONTROL_RUN
0
The download is allowed to run.
Default value.
Downloads.CONTROL_PAUSED
0
The download is paused.
Status Values
<>
<>
Constants name
Constant value
Description
Notes
Downloads.STATUS_PENDING
190
Download hasn't started.
Downloads.STATUS_PENDING_PAUSED
191
Download hasn't started and can't start immediately (network unavailable, paused by user).
Not rently used.
Downloads.STATUS_RUNNING
192
Download has started and is running
Downloads.STATUS_RUNNING_PAUSED
193
Download has started, but can't run at the moment (network unavailable, paused by user).
Downloads.STATUS_SUCCESS
200
Download completed successfully.
Downloads.STATUS_BAD_REQUEST
400
Couldn't initiate the request, or server response 400.
Downloads.STATUS_NOT_ACCEPTABLE
406
No handler to view the file (external downloads), or server response 406.
External downloads are nt to be user-visible, and are aborted if there's no application to handle the relevant MIME type.
Downloads.STATUS_LENGTH_REQUIRED
411
The download manager can't know the length of the download.
Because of the unreliability of cell works, the download manager only performs downloads when it can verify that it has received all the data for a download, except if the initiating app sets the Downloads.NO_INTEGRITY flag.
Downloads.STATUS_PRECONDITION_FAILED
412
The download manager can't resume an interrupted download, because it didn't receive enough information m the server to be able to resume.
Downloads.STATUS_CANCELED
490
The download was canceled by a cause outside the Download Manager.
Formerly known as Downloads.TUS_CANCELLED. Might be impossible to observe in 1.0.
Downloads.STATUS_UNKNOWN_ERROR
491
The download was aborted because of an unknown error.
Formerly known as Downloads.STATUS_ERROR. Typically the result of a runtime exception that is not explicitly handled.
Downloads.STATUS_FILE_ERROR
492
The download was aborted because the data couldn't be saved.
Most commonly happens when the filesystem is l.
Downloads.STATUS_UNHANDLED_REDIRECT
493
The download was aborted because the server returned a redirect code 3xx that the download manager doesn't dle.
The download manager currently handles 301, 302 and 307.
Downloads.STATUS_TOO_MANY_REDIRECTS
494
The download was aborted because the download manager received too many redirects while trying to find the ual file.
Downloads.STATUS_UNHANDLED_HTTP_CODE
495
The download was aborted because the server returned a status code that the download manager doesn't handle.td>
Standard codes 3xx, 4xx and 5xx don't trigger this.
Downloads.STATUS_HTTP_DATA_ERROR
496
The download was aborted because of an unrecoverable error trying to get data over the network.
ically this happens when the download manager received several I/O Exceptions in a row while the network is available and without being able to download any data.
4xx
standard HTTP/1.1 4xx codes returned by the server are used as-is.
Don't rely on non-standard values being passed through, especially the her values that would collide with download manager codes.
5xx
standard HTTP/1.1 5xx codes returned by the server are used as-is.
Don't rely on non-standard values being passed through.
Status Helper Functions
Function signature
Description
public static boolean Downloads.isStatusInformational(int status)
Returns whether the status code matches a download that hasn't completed yet.
public static boolean Downloads.isStatusSuspended(int status)
Returns whether the download hasn't completed yet but isn't currently making progress.
public static boolean Downloads.isStatusSuccess(int status)
Returns whether the download successfully completed.
public static boolean Downloads.isStatusError(int status)
Returns whether the download failed.
public static boolean Downloads.isStatusClientError(int status)
Returns whether the download failed because of a client error.
public static boolean Downloads.isStatusServerError(int status)
Returns whether the download failed because of a server error.
public static boolean Downloads.isStatusCompleted(int status)
Returns whether the download completed (with no distinction between success and failure).
Intents
The download manager sends an intent broadcast Downloads.DOWNLOAD_COMPLETED_ACTION when a download completes.
The download manager sends an intent broadcast Downloads.NOTIFICATION_CLICKED_ACTION when the user clicks a download notification that doesn't match a download that can be opened (e.g. because notification is for several downloads at a time, or because it's for an incomplete download, or because it's for a private download).
The download manager starts an activity with Intent.ACTION_VIEW when the user clicks a download notification that matches a download that can be opened.
Differences between 1.0 and Cupcake
WARNING These are the differences for apps built from source in the Android source tree, i.e. they don't cover any of the cases of binary incompatibility.
Cursors returned by the content provider are now read-only.
SQL "where" statements are now verified and must follow a rigid syntax (the most visible aspect being that all parameters much be single-quoted).
Columns and constants that were unused or ineffective are gone: METHOD, NO_SYSTEM_FILES, DESTINATION_DATA_CACHE, OTA_UPDATE.
OTHER_UID and DESTINATION_CACHE_PARTITION require android.permission.ACCESS_DOWNLOAD_MANAGER_ADVANCED.
The list of columns that can be queried and read is limited: _ID, APP_DATA, _DATA, MIMETYPE, CONTROL, STATUS, T_MODIFICATION, NOTIFICATION_PACKAGE, NOTIFICATION_CLASS, TOTAL_BYTES, CURRENT_BYTES, TITLE, DESCRIPTION.
The list of columns that can be initialized by apps is limited: URI, APP_DATA, NO_INTEGRITY, FILENAME_HINT, MIMETYPE, DESTINATION, VISIBILITY, CONTROL, STATUS, LAST_MODIFICATION, NOTIFICATION_PACKAGE, NOTIFICATION_CLASS, NOTIFICATION_EXTRAS, COOKIE_DATA, USER_AGENT, REFERER, OTHER_UID, TITLE, DESCRIPTION.
The list of columns that can be updated by apps is limited: APP_DATA, VISIBILITY, CONTROL, TITLE, DESCRIPTION.
Downloads to the SD card default to have notifications that are visible after completion, internal downloads default to notifications that are always hidden.
The constant FILENAME was renamed Downloads._DATA.
Downloads can be paused and resumed by writing to the CONTROL column. Downloads.CONTROL_RUN (Default) makes the download go, Downloads.CONTROL_PAUSED pauses it.
New column APP_DATA that is untouched by the download manager, used to store app-specific info about the downloads.
Minor differences unlikely to affect applications:
The notification class/package must now match the UID.
The backdoor to see the entire provider (which was intended to implement UIs) is gone.
Private column names were removed from the public API.
Writing code that works on both the 1.0 and Cupcake versions of the download manager.
If you're not 100% sure that you need to be reading this chapter, don't read it.
Basic rule: don't use features that only exist in one of the two implementations (POST, downloads to /data...).
Also, don't use columns in 1.0 that are protected or hidden in Cupcake.
Unfortunately, that's not always entirely possible.
Areas of concern:
Some columns were renamed. FILENAME became Downloads._DATA ("_data"), and ENTITY became APP_DATA ("entity").
The difference can be used to distinguish between 1.0 and Cupcake, though reflection.
The difference prevents from using any of the symbolic constants directly in source code: if the same binary wants to run on both 1.0 and Cupcake, it will have to hard-code the values of those stants or use reflection to get to them.
URI column accessible in 1.0 but protected in Cupcake. Code that relies on being able to re-read its URI should be using APP_DATA in Cupcake, but that column doesn't exist as such in 1.0.
If the code detects that it's running on Cupcake, write URI to APP_DATA in addition to URI, and query and read from the appropriate column.
Since the underlying column for APP_DATA exists in both 1.0 and Cupcake even though it has different names, it can actually be used in both cases (see note above about renamed columns).
Some of the error codes have been renumbered. STATUS_UNHANDLED_HTTP_CODE and STATUS_HTTP_DATA_ERROR were bumped up from 494 and 495 to 495 and 496.
Backward compatibility is not guaranteed: the download manager APIs weren't meant to be backward compatible yet. As such it's impossible to guarantee that code that uses the Cupcake download manager l be binary-compatible with future versions.
I intend to eventually change the column name for APP_DATA to "app_data". Because of that, code should use reflection to get to that name instead of hard-coding "entity", so that it always gets the ht value for the string.
I intend to refine the handling of filenames and content URIs, exposing separate columns for situations where a download can be accessed both as a file and through a content URI (e.g. stuff that is ognized by the media scanner). Unfortunately at this point this feature isn't clear in my mind. I'd recommend using reflection to look for the Downloads._DATA column, and if it isn't there to look for the ENAME column (which has the advantage of also dealing with the difference between 1.0 and Cupcake).
I intend to renumber the error codes, especially those in the 4xx range, and especially those below 490 (which overlap with standard HTTP error codes but will probably be separated). Reflection would rove the probability to getting to them in the future. Unfortunately, the names of the constants are likely to change in the process, in order to disambiguate codes coming from HTTP from those generated ally. I might try to stick to the following pattern: where a constant is currently named STATUS_XXX, its locally-generated version in the future might be named STATUS_LOCAL_XXX while the current constant e might disappear. Using reflection to try to get to the possible new name instead of using the old name might improve the probability of compatibility in the future. That being said, it is critically ortant to properly handle the full ranges or error codes, especially the 4xx range, as "expected" errors, and it is far preferable to not try to distinguish between those codes at all: use the functions nloads.isError and Downloads.isClientError to easily recognize those entire ranges. In order of probability, the 1xx range is the second most likely to be affected.
Functional Specification
All the details about what the product does.
TODO
Release notes for Cupcake
HTTP codes 301, 302, 303 and 307 (redirects) are now supported
HTTP code 503 is now handled, with support for retry-after in delay-seconds
Downloads that were cleanly interrupted are now resumed instead of failing
Applications can now pause their downloads
Retry delays are now randomized
Connectivity is now checked on all interfaces
Downloads with invalid characters in file name can now be saved
The download manager is built primarily around a ContentProvider and a Service. The ContentProvider part is the front end, i.e. applications communicate with the download manager through the provider. The vice part is the back end, which contains the actual download logic, running as a background process.
As a first approach, the provider is essentially a canonical provider backed by a SQLite3 database. The biggest difference between the download provider and a "plain" provider is that the download provider ressively validates its inputs, for security reasons.
The service is a background process that performs the actual downloads as requested by the applications. The service doesn't offer any bindable interface, the service object exists strictly so that the tem knows how to prioritize the download manager's process against other processes when memory is tight.
Communication between the provider and the service is done through public Android APIs, so that the two components are deeply decoupled (they could in fact run in different processes). The download manager rts the service whenever a change is made that can start or restart a download. The service observes and queries the provider for changes, and updates the provider as the download progresses.
There are a few secondary classes that provide auxiliary functions.
A Receiver listens to several broadcasts. Is receives some system broadcasts when the system boots (so that the download manager can resume downloads that were interrupted when the system was turned off) or n the connectivity changes (so that the download manager can restart downloads that were interrupted when connectivity was lost). It also receives intents when the user selects a download notification.
Finally, some helper classes provide support functions.
Most significantly, DownloadThread is responsible for performing the actual downloads as part of the DownloadService's functionality, while UpdateThread is responsible for updating the DownloadInfo whenever DownloadProvider data changes.
DownloadInfo and DownloadFileInfo hold pure data structures, with little or no actual logic.
Lexer takes care of validating the snippets of SQL data that are received from applications, to avoid cases of SQL injection.
The service keeps a copy of the provider data in RAM, so that it can determine what changed in the provider when it receives a change notification through the ContentObserver. That data is kept in an array DownloadInfo structures.
Each DownloadThread performs the operations for a single download (or, more precisely, for a single HTTP transaction). Each DownloadThread is backed by a DownloadInfo object. which is in fact on of the ects kept by the DownloadService. While a download is running, the DownloadService can influence the download by writing data into the relevant DownloadInfo object, and the DownloadThread checks that object appropriate times during the download.
Because the DownloadService updates the DownloadInfo objects asynchronously from everything else (it uses a dedicated thread for that purpose), a lot of care has to be taken when upgrading the DownloadInfo ect. In fact, only the DownloadService's updateThread function can update that object, and it should be considered read-only to every other bit of code. Even within the updateThread function, some care must taken to ensure that the DownloadInfos don't get out of sync with the provider.
On the other hand, the DownloadService's updateThread function does upgrade the DownloadInfo when it spawns new DownloadThreads (and in a few more circumstances), and when it does that it must also update DownloadProvider (or risk seeing its DownloadInfo data get overwritten).
Because of all that, all code outside of the DowloadService's updateThread must neither read from DownloadProvider nor write to the DownloadInfo objects under any circumstances. The DownloadService's ateFunction is responsible for copying data from the DownloadProvider to the DownloadInfo objects, and must ensure that the DownloadProvider remains in sync with the information it writes into the nloadInfo objects.
Implementation Documentation
How individual modules are implemented.
TODO To be completed
Database formats
Android 1.0, format version 31, table downloads:
Column
Type
"_id"
INTEGER PRIMARY KEY AUTOINCREMENT
"uri"
TEXT
"method"
INTEGER
"entity"
TEXT
"no_integrity"
BOOLEAN
"hint"
TEXT
"otaupdate"
BOOLEAN
"_data"
TEXT
"mimetype"
TEXT
"destination"
INTEGER
"no_system"
BOOLEAN
"visibility"
INTEGER
"control"
INTEGER
"status"
INTEGER
"numfailed"
INTEGER
"lastmod"
BIGINT
"notificationpackage"
TEXT
"notificationclass"
TEXT
"notificationextras"
TEXT
"cookiedata"
TEXT
"useragent"
TEXT
"referer"
TEXT
"total_bytes"
INTEGER
"current_bytes"
INTEGER
"etag"
TEXT
"uid"
INTEGER
"otheruid"
INTEGER
"title"
TEXT
"description"
TEXT
"scanned"
BOOLEAN
Cupcake, format version 100: Same as format version 31.
Future Directions
WARNING This section is for informative purposes only.
API
Expose Download Manager to 3rd party apps - security, robustness .
Validate application-provided user agent, cookies, etc... to protect against e.g. header injection.
Allow trust-and-verify MIME type.
Extract response string from HTTP responses, extract entity from failed responses
If app fails to be notified, retry later - don't give up because of a single failure.
Support data: URIs .
Download files to app-provided content provider .
Don't pass HTTP codes above about 490 to the initiating app (figure out what the threshold should be).
Allow initiating app to specify that it wants wifi-only downloads .
Provide SQL "where" clauses for the different categories of status codes, matching isStatusXXX().
There has to be a mechanism by which old downloads are automatically purged from /cache if there's not enough space .
Clicking the notification for a completed download should go through the initiating app instead of directly opening the file (but what about fire and forget?).
Clean up the difference between pending_network (waiting for network) and running_paused (user paused the download).
Allow any app to access the downloaded file (but no other column) of user-downloaded files .
Provider should return more errors and throw fewer exceptions if possible.
Add option to auto-dismiss notifications for completed downloads after a given time .
Delete filename in case of failure - better handle the separation between filename and content URI.
Allow the initiating application to specify that it wants to restart downloads from scratch if they're interrupted and can't be resumed .
Save images directly from browser without re-downloading data .
Give applications the ability to explicitly specify the full target filename (not just a hint).
Give applications the ability to download multiple files into multiple set locations as if they were a single "package".
Give applications the ability to download files only if they haven't been already downloaded.
Give applications the ability to download files that have already been downloaded only if there's a newer version.
Set-cookie in the response.
basic auth .
app-provided prompts for basic auth, ssl, redirects.
File should be hidden from initiating application when DRM.
Delay writing of app-visible filename column until file visible (split user-visible vs private files?).
Separate locally-generated status codes from standard HTTP codes.
Allow app to specify it doesn't want to resume downloads across reboots (because they might require additional work).
Allow app to prioritize user-initiated downloads.
Allow app to specify length of download (full trust, or trust-and-verify).
Support POST.
Support PUT.
Support plugins for additional protocols and download descriptors.
Rename columns to have an appropriate COLUMN_ prefix.
HTTP Handling
Fail download immediately on authoritative unresolved hostnames .
The download manager should use the browser's user-agent by default.
Redirect with HTTP Refresh headers (download current and target content with non-zero refresh).
Handle content-encoding header.
Handle transfer-encoding header.
Find other ways to validate interrupted downloads (signature, last-mod/if-modified-since) .
Make downloads time out in case of long time with no activity.
File names
Protect against situations where there's already a "downloads" directory on SD card.
Deal with filenames with invalid characters.
Refine the logic that builds filenames to better match desktop browsers - drop the query string.
URI-decode filenames generated from URIs.
Better deal with filenames that end in '.'.
Deal with URIs that end in '/' or '?'.
Investigate how to better deal with filenames that have multiple extensions.
UI
Prompt for redirects across domains or cancel.
Prompt for redirects from SSL or cancel.
Prompt for basic auth or cancel.
Prompt for SSL with untrusted/invalid/expired certificates or cancel.
Reduce number of icons in the title bar, possibly as low as 1 (animated if there are ongoing downloads, fixed if all downloads have completed) .
UI to cancel visible downloads.
UI to pause visible downloads.
Reorder downloads.
View SSL certificates.
Indicate secure downloads.
Handling of specific MIME types
Parse HTML for redirects with meta tag.
Handle charsets and transcoding of text files.
Deal with multiparts.
Support OMA downloads with DD and data in same multipart, i.e. combined delivery.
Assume application/octet-stream for http responses with no mime type.
Download anything if an app supports application/octet-stream.
Download any text/* if an application supports text/plain.
Should the media scanner be invoked on DRM downloads?
Refresh header with timer should be followed if content is not downloadable.
Support OMA downloads.
Support MIDP-OTA downloads.
Support Sprint MCD downloads.
Sniff content when receiving MIME-types known to be inaccurately sent by misconfigured servers.
Management of downloads based on environment
If the device routinely connects over wifi, delay non-interactive downloads by a certain amount of time in case wifi becomes available
Turn on wifi if possible
Fall back to cell when wifi is available but download doesn't proceed
Be smarter about spurious losses (i.e. exceptions while network appears up) when the active network changes (e.g. turn on wifi while downloading over cell).
Investigate the use of wifi locks, especially when performing non-resumable downloads.
Poll network state (and maybe even try to connect) even without notifications from the connectivity manager (in case the notifications go AWOL or get inconsistent) .
Pause when conditions degrade .
Pause when roaming.
Throttle or pause when user is active.
Pause on slow networks (2G).
Pause when battery is low.
Throttle to not overwhelm the link.
Pause when sync is active.
Deal with situations where the active connection is down but there's another connection available
Download files at night when the user is not explicitly waiting.
Management of simultaneous downloads
Pipeline requests on limited number of sockets, run downloads sequentially .
Manage bandwidth to not starve foreground tasks.
Run unsized downloads on their own (on a per-filesystem basis) to avoid failing multiple of them because of a full filesystem .
Minor functional changes, edge cases
The database could be somewhat checked when it's opened.
[DownloadProvider.java] When upgrading the database, the numbering of ids should restart where it left off.
[DownloadProvider.java] Handle errors when failing to start the service.
[DownloadProvider.java] Explicitly populate all database columns that have documented default values, investigate whether that can be done at the SQL level.
[DownloadProvider.java] It's possible that the last update time should be updated by the Sevice logic, not by the content provider.
When relevant, combine logged messages on fewer lines.
[DownloadService.java] Trim the database in the provider, not in the service. Notify application when trimming. Investigate why the row count seems off by one. Enforce on an ongoing basis.
[DownloadThread.java] When download is restarted and MIME type wasn't provided by app, don't re-use MIME type.
[DownloadThread.java] Deal with mistmatched file data sizes (between database and filesystem) when resuming a download, or with missing files that should be here.
[DownloadThread.java] Validate that the response content-length can be properly parsed (i.e. presence of a string doesn't guarantee correctness).
[DownloadThread.java] Be finer-grained with the way file permissions are managed in /cache - don't 0644 everything .
Truncate files before deleting them, in case they're still open cross-process.
Restart from scratch downloads that had very little progress .
Deal with situations where /data is full as it prevents the database from growing (DoS) .
Wait until file scanned to notify that download is completed.
Missing some detailed logging about IOExceptions.
Allow to disable LOGD debugging independently from system setting.
Pulling the battery during a download corrupts files (lots of zeros written) .
Should keep a bit of "emergency" database storage to initiate the download of an OTA update, in a file that is pre-allocated whenever possible (how to know it's an OTA update?).
Figure out how to hook up into dumpsys and event log.
Use the event log to log download progress.
Use /cache to stage downloads that eventually go to the sd card, to avoid having sd files open too long in case the use pulls the card and to avoid partial files for too long.
Maintain per-application usage statistics.
There might be corner cases where the notifications are slightly off because different notifications might be using PendingIntents that can't be distinguished (differing only by their extras).
Architecture and Implementation
The content:// Uri of individual downloads could be cached instead of being re-built whenever it's needed.
[DownloadProvider.java] Solidify extraction of id from URI
[DownloadProvider.java] Use ContentURIs.parseId(uri) to extra the id from various functions.
[DownloadProvider.java] Use StringBuilder to build the various queries.
[DownloadService.java] Cache interface to the media scanner service more aggressively.
[DownloadService.java] Investigate why unbinding from the media scanner service sometimes throws an exception
[DownloadService.java] Handle exceptions in the service's UpdateThread - mark that there's no thread left.
[DownloadService.java] At the end of UpdateThread, closing the cursor should be done even if there's an exception. Also log the exception, as we'd be in an inconsistent state.
[DownloadProvider.java] Investigate whether the download provider should aggressively cache the result of getContext() and getContext().getContentResolver()
Document the database columns that are most likely to stay unchanged throughout versions, to increase the chance being able to perform downgrades.
[DownloadService.java] Sanity-check the ordering of the local cache when adding/removing rows.
[DownloadService.java] Factor the code that checks for DRM types into a separate function.
[DownloadService.java] Factor the code that notifies applications into a separate function (codepath with early 406 failure)
[DownloadService.java] Check for errors when spawning download threads.
[DownloadService.java] Potential race condition when a download completes at the same time as it gets deleted through the content provider - see deleteDownload().
[DownloadService.java] Catch all exceptions in scanFile - don't trust a remote process to the point where we'd let it crash us.
[DownloadService.java] Limit number of attempts to scan a file.
[DownloadService.java] Keep less data in RAM, especially about completed downloads. Investigating cutting unused columns if necessary
[DownloadThread.java] Don't let exceptions out of run() - that'd kill the service, which'd accomplish no good.
[DownloadThread.java] Keep track of content-length responses in a long, not in a string that we keep parsing .
[DownloadThread.java] Use variable-size buffer to avoid thousands of operations on large downloads
[Helpers.java] Deal with atomicity of checking/creating file.
[Helpers.java] Handle differences between content-location separators and filesystem separators.
Optimize database queries: use projections to reduce number of columns and get constant column numbers.
Index last-mod date in DB, because of ordered searches. Investigate whether other columns need to be indexed (Hidden?)
Deal with the fact that sqlite INTEGER matches java long (63-bit) .
Use a single HTTP client for the entire download manager.
Could use fewer alarms - currently setting new alarm each time database updated .
Obsolete columns should be removed from the database .
Assign relevant names to threads.
Investigate and handle the different subclasses of IOException appropriately .
There's potentially a race condition around read-modify-write cycles in the database, between the Service's updateFromProvider thread and the worker threads (and possibly more). Those should be hronized appropriately, and the provider should be hardened to prevent asynchronous changes to sensitive data (or to synchronize when there's no other way, though I'd rather avoid that) .
Temporary file leaks when downloads are deleted while the service isn't running .
Increase priority of updaterThread while in the critical section (to avoid issues of priority inheritance with the main thread).
Explicitly specify which interface to use for a given download (to get better sync with the connection manager).
Cancel the requests on more kinds of errors instead of trusting the garbage collector.
Issues with the fact that parseInt can throw exceptions on invalid server headers.
Code style, refactoring
Fix lint warnings
Make sure that comments fit in 80 columns to match style guide
Unify code style when dealing with lines longer than 100 characters
[Constants.java] constants should be organized by logical groups to improve readability.
Use fewer internal classes (Helpers, Constants...) .
Browser changes
Move download UI outside of browser, so that browser doesn't need to access the provider.
Make browser sniff zips and jars to see if they're apks.
Live handoff of browser-initiated downloads (download in browser, browser update download UI, hand over to download manager on retry).
When developing a component, for example a C++ shared library, it must be
possible to easily rebuild just that component, and not have to wait more than a
couple seconds for dependency checks, and not have to wait for unneeded
components to be built.
Both Environment and Config File Based Settings
To set the target, and other options, some people on the team like to have a
configuration file in a directory so they do not have an environment setup
script to run, and others want an environment setup script to run so they can
run builds in different terminals on the same tree, or switch back and forth
in one terminal. We will support both.
Object File Directory / make clean
Object files and other intermediate files will be generated into a directory
that is separate from the source tree. The goal is to have make clean be
"rm -rf " in the tree root directory. The primary goals of
this are to simplify searching the source tree, and to make "make clean" more
reliable.
SDK
The SDK will be a tarball that will allow non-OS-developers to write apps.
The apps will actually be built by first building the SDK, and then building
the apps against that SDK. This will hopefully (1) make writing apps easier
for us, because we won't have to rebuild the OS as much, and we can use the
standard java-app development tools, and (2) allow us to dog-food the SDK, to
help ensure its quality. Cedric has suggested (and I agree) that apps built
from the SDK should be built with ant. Stay tuned for more details as we
figure out exactly how this will work.
Dependecies
Dependencies should all be automatic. Unless there is a custom tool involved
(e.g. the webkit has several), the dependencies for shared and static libraries,
.c, .cpp, .h, .java, java libraries, etc., should all work without intervention
in the Android.mk file.
Hiding command lines
The default of the build system will be to hide the command lines being
executed for make steps. It will be possible to override this by specifying
the showcommands pseudo-target, and possibly by setting an environment
variable.
Wildcard source files
Wildcarding source file will be discouraged. It may be useful in some
scenarios. The default $(wildcard *) will not work due to the
current directory being set to the root of the build tree.
Multiple targets in one directory
It will be possible to generate more than one target from a given
subdirectory. For example, libutils generates a shared library for the target
and a static library for the host.
Makefile fragments for modules
Android.mk is the standard name for the makefile fragments that
control the building of a given module. Only the top directory should
have a file named "Makefile".
Use shared libraries
Currently, the simulator is not built to use shared libraries. This should
be fixed, and now is a good time to do it. This implies getting shared
libraries to work on Mac OS.
Nice to Have
These things would be nice to have, and this is a good place to record them,
however these are not promises.
Simultaneous Builds
The hope is to be able to do two builds for different combos in the same
tree at the same time, but this is a stretch goal, not a requirement.
Doing two builds in the same tree, not at the same time must work. (update:
it's looking like we'll get the two builds at the same time working)
Deleting headers (or other dependecies)
Problems can arise if you delete a header file that is referenced in
".d" files. The easy way to deal with this is "make clean". There
should be a better way to handle it. (from fadden)
One way of solving this is introducing a dependency on the directory. The
problem is that this can create extra dependecies and slow down the build.
It's a tradeoff.
Multiple builds
General way to perform builds across the set of known platforms. This
would make it easy to perform multiple platform builds when testing a
change, and allow a wide-scale "make clean". Right now the buildspec.mk
or environment variables need to be updated before each build. (from fadden)
Aftermarket Locales and Carrier
We will eventually need to add support for creating locales and carrier
customizations to the SDK, but that will not be addressed right now.
If you source the file build/envsetup.sh into your bash environment,
. build/envsetup.shyou'll get a few helpful shell functions:
printconfig - Prints the current configuration as set by the
lunch and choosecombo commands.
m - Runs make from the top of the tree. This is
useful because you can run make from within subdirectories. If you have the
TOP environment variable set, it uses that. If you don't, it looks
up the tree from the current directory, trying to find the top of the tree.
croot - cd to the top of the tree.
sgrep - grep for the regex you provide in all .c, .cpp, .h, .java,
and .xml files below the current directory.
Build flavors/types
When building for a particular product, it's often useful to have minor
variations on what is ultimately the final release build. These are the
currently-defined "flavors" or "types" (we need to settle on a real name
for these).
eng
This is the default flavor. A plain "make" is the
same as "make eng". droid is an alias
for eng.
Installs non-APK modules that have no tags specified.
Installs APKs according to the product definition files, in
addition to tagged APKs.
ro.secure=0
ro.debuggable=1
ro.kernel.android.checkjni=1
adb is enabled by default.
user
"make user"
This is the flavor intended to be the final release bits.
Installs modules tagged with shell_$(TARGET_SHELL) and user.
Installs non-APK modules that have no tags specified.
Installs APKs according to the product definition files; tags
are ignored for APK modules.
ro.secure=1
ro.debuggable=0
adb is disabled by default.
userdebug
"make userdebug"
The same as user, except:
Also installs modules tagged with debug.
ro.debuggable=1
adb is enabled by default.
If you build one flavor and then want to build another, you should run
"make installclean" between the two makes to guarantee that
you don't pick up files installed by the previous flavor. "make
clean" will also suffice, but it takes a lot longer.
More pseudotargets
Sometimes you want to just build one thing. The following pseudotargets are
there for your convenience:
droid - make droid is the normal build. This target
is here because the default target has to have a name.
all - make all builds everything make
droid does, plus everything whose LOCAL_MODULE_TAGS do not
include the "droid" tag. The build server runs this to make sure
that everything that is in the tree and has an Android.mk builds.
clean-$(LOCAL_MODULE) and clean-$(LOCAL_PACKAGE_NAME) -
Let you selectively clean one target. For example, you can type
make clean-libutils and it will delete libutils.so and all of the
intermediate files, or you can type make clean-Home and it will
clean just the Home app.
clean - make clean deletes all of the output and
intermediate files for this configuration. This is the same as rm -rf
out/<configuration>/
clobber - make clobber deletes all of the output
and intermediate files for all configurations. This is the same as
rm -rf out/.
dataclean - make dataclean deletes contents of the data
directory inside the current combo directory. This is especially useful on the
simulator and emulator, where the persistent data remains present between
builds.
showcommands - showcommands is a modifier target
which causes the build system to show the actual command lines for the build
steps, instead of the brief descriptions. Most people don't like seeing the
actual commands, because they're quite long and hard to read, but if you need
to for debugging purposes, you can add showcommands to the list
of targets you build. For example make showcommands will build
the default android configuration, and make runtime showcommands
will build just the runtime, and targets that it depends on, while displaying
the full command lines. Please note that there are a couple places where the
commands aren't shown here. These are considered bugs, and should be fixed,
but they're often hard to track down. Please let
android-build-team know if you find
any.
LOCAL_MODULE - Anything you specify as a LOCAL_MODULE
in an Android.mk is made into a pseudotarget. For example, make
runtime might be shorthand for make
out/linux-x86-debug/system/bin/runtime (which would work), and
make libkjs might be shorthand for make
out/linux-x86-debug/system/lib/libkjs.so (which would also work).
targets - make targets will print a list of all of
the LOCAL_MODULE names you can make.
Use the templates/shared_library file, or the
templates/shared_library_host file.
Remember that on the target, we use shared libraries, and on the host,
we use static libraries, since executable size isn't as big an issue, and it
simplifies distribution in the SDK.
Static Libraries
Use the templates/static_library file, or the
templates/static_library_host file.
Remember that on the target, we use shared libraries, and on the host,
we use static libraries, since executable size isn't as big an issue, and it
simplifies distribution in the SDK.
ifeq ($(TARGET_BUILD_TYPE),release)
LOCAL_CFLAGS += -DNDEBUG=1
endif
# from libutils
ifeq ($(TARGET_OS),linux)
# Use the futex based mutex and condition variable
# implementation from android-arm because it's shared mem safe
LOCAL_SRC_FILES += futex_synchro.c
LOCAL_LDLIBS += -lrt -ldl
endif
INTERNAL_ - These variables are critical to functioning of
the build system, so you shouldn't create variables named like this, and
you probably shouldn't be messing with these variables in your makefiles.
HOST_ and TARGET_ - These contain the directories
and definitions that are specific to either the host or the target builds.
Do not set variables that start with HOST_ or TARGET_ in your makefiles.
BUILD_ and CLEAR_VARS - These contain the names of
well-defined template makefiles to include. Some examples are CLEAR_VARS
and BUILD_HOST_PACKAGE.
Any other name is fair-game for you to use in your Android.mk. However,
remember that this is a non-recursive build system, so it is possible that
your variable will be changed by another Android.mk included later, and be
different when the commands for your rule / module are executed.
LOCAL_ASSET_FILES
In Android.mk files that include $(BUILD_PACKAGE) set this
to the set of files you want built into your app. Usually:
LOCAL_ASSET_FILES += $(call find-subdir-assets)
This will probably change when we switch to ant for the apps' build
system.
LOCAL_CC
If you want to use a different C compiler for this module, set LOCAL_CC
to the path to the compiler. If LOCAL_CC is blank, the appropriate default
compiler is used.
LOCAL_CXX
If you want to use a different C++ compiler for this module, set LOCAL_CXX
to the path to the compiler. If LOCAL_CXX is blank, the appropriate default
compiler is used.
LOCAL_CFLAGS
If you have additional flags to pass into the C or C++ compiler, add
them here. For example:
LOCAL_CFLAGS += -DLIBUTILS_NATIVE=1
LOCAL_CPPFLAGS
If you have additional flags to pass into only the C++ compiler, add
them here. For example:
LOCAL_CPPFLAGS += -ffriend-injection
LOCAL_CPPFLAGS is guaranteed to be after LOCAL_CFLAGS
on the compile line, so you can use it to override flags listed in
LOCAL_CFLAGS.
LOCAL_CPP_EXTENSION
If your C++ files end in something other than ".cpp",
you can specify the custom extension here. For example:
LOCAL_CPP_EXTENSION := .cc
Note that all C++ files for a given module must have the same
extension; it is not currently possible to mix different extensions.
LOCAL_NO_DEFAULT_COMPILER_FLAGS
Normally, the compile line for C and C++ files includes global include
paths and global cflags. If LOCAL_NO_DEFAULT_COMPILER_FLAGS
is non-empty, none of the default includes or flags will be used when compiling
C and C++ files in this module.
LOCAL_C_INCLUDES, LOCAL_CFLAGS, and
LOCAL_CPPFLAGS will still be used in this case, as will
any DEBUG_CFLAGS that are defined for the module.
LOCAL_COPY_HEADERS
This will be going away.
The set of files to copy to the install include tree. You must also
supply LOCAL_COPY_HEADERS_TO.
This is going away because copying headers messes up the error messages, and
may lead to people editing those headers instead of the correct ones. It also
makes it easier to do bad layering in the system, which we want to avoid. We
also aren't doing a C/C++ SDK, so there is no ultimate requirement to copy any
headers.
LOCAL_COPY_HEADERS_TO
This will be going away.
The directory within "include" to copy the headers listed in
LOCAL_COPY_HEADERS to.
This is going away because copying headers messes up the error messages, and
may lead to people editing those headers instead of the correct ones. It also
makes it easier to do bad layering in the system, which we want to avoid. We
also aren't doing a C/C++ SDK, so there is no ultimate requirement to copy any
headers.
LOCAL_C_INCLUDES
Additional directories to instruct the C/C++ compilers to look for header
files in. These paths are rooted at the top of the tree. Use
LOCAL_PATH if you have subdirectories of your own that you
want in the include paths. For example:
You should not add subdirectories of include to
LOCAL_C_INCLUDES, instead you should reference those files
in the #include statement with their subdirectories. For
example:
#include <utils/KeyedVector.h>
not #include <KeyedVector.h>
There are some components that are doing this wrong, and should be cleaned
up.
LOCAL_MODULE_TAGS
Set LOCAL_MODULE_TAGS to any number of whitespace-separated
tags. If the tag list is empty or contains droid, the module
will get installed as part of a make droid. Modules with the tag
shell_$(TARGET_SHELL) will also be installed. Otherwise, it will
only get installed by running make <your-module>
or with the make all pseudotarget.
LOCAL_REQUIRED_MODULES
Set LOCAL_REQUIRED_MODULES to any number of whitespace-separated
module names, like "libblah" or "Email". If this module is installed, all
of the modules that it requires will be installed as well. This can be
used to, e.g., ensure that necessary shared libraries or providers are
installed when a given app is installed.
LOCAL_FORCE_STATIC_EXECUTABLE
If your executable should be linked statically, set
LOCAL_FORCE_STATIC_EXECUTABLE:=true. There is a very short
list of libraries that we have in static form (currently only libc). This is
really only used for executables in /sbin on the root filesystem.
LOCAL_GENERATED_SOURCES
Files that you add to LOCAL_GENERATED_SOURCES will be
automatically generated and then linked in when your module is built.
See the Custom Tools template makefile for an
example.
LOCAL_JAVACFLAGS
If you have additional flags to pass into the javac compiler, add
them here. For example:
LOCAL_JAVACFLAGS += -Xlint:deprecation
LOCAL_JAVA_LIBRARIES
When linking Java apps and libraries, LOCAL_JAVA_LIBRARIES
specifies which sets of java classes to include. Currently there are
two of these: core and framework.
In most cases, it will look like this:
LOCAL_JAVA_LIBRARIES := core framework
Note that setting LOCAL_JAVA_LIBRARIES is not necessary
(and is not allowed) when building an APK with
"include $(BUILD_PACKAGE)". The appropriate libraries
will be included automatically.
LOCAL_LDFLAGS
You can pass additional flags to the linker by setting
LOCAL_LDFLAGS. Keep in mind that the order of parameters is
very important to ld, so test whatever you do on all platforms.
LOCAL_LDLIBS
LOCAL_LDLIBS allows you to specify additional libraries
that are not part of the build for your executable or library. Specify
the libraries you want in -lxxx format; they're passed directly to the
link line. However, keep in mind that there will be no dependency generated
for these libraries. It's most useful in simulator builds where you want
to use a library preinstalled on the host. The linker (ld) is a particularly
fussy beast, so it's sometimes necessary to pass other flags here if you're
doing something sneaky. Some examples:
If your package doesn't have a manifest (AndroidManifest.xml), then
set LOCAL_NO_MANIFEST:=true. The common resources package
does this.
LOCAL_PACKAGE_NAME
LOCAL_PACKAGE_NAME is the name of an app. For example,
Dialer, Contacts, etc. This will probably change or go away when we switch
to an ant-based build system for the apps.
LOCAL_PATH
The directory your Android.mk file is in. You can set it by putting the
following as the first line in your Android.mk:
LOCAL_PATH := $(my-dir)
The my-dir macro uses the
MAKEFILE_LIST
variable, so you must call it before you include any other makefiles. Also,
consider that any subdirectories you inlcude might reset LOCAL_PATH, so do your
own stuff before you include them. This also means that if you try to write
several include lines that reference LOCAL_PATH,
it won't work, because those included makefiles might reset LOCAL_PATH.
LOCAL_POST_PROCESS_COMMAND
For host executables, you can specify a command to run on the module
after it's been linked. You might have to go through some contortions
to get variables right because of early or late variable evaluation:
When including $(BUILD_PREBUILT) or $(BUILD_HOST_PREBUILT), set these to
executables that you want copied. They're located automatically into the
right bin directory.
LOCAL_PREBUILT_LIBS
When including $(BUILD_PREBUILT) or $(BUILD_HOST_PREBUILT), set these to
libraries that you want copied. They're located automatically into the
right lib directory.
LOCAL_SHARED_LIBRARIES
These are the libraries you directly link against. You don't need to
pass transitively included libraries. Specify the name without the suffix:
The build system looks at LOCAL_SRC_FILES to know what source
files to compile -- .cpp .c .y .l .java. For lex and yacc files, it knows
how to correctly do the intermediate .h and .c/.cpp files automatically. If
the files are in a subdirectory of the one containing the Android.mk, prefix
them with the directory name:
LOCAL_SRC_FILES := \
file1.cpp \
dir/file2.cpp
LOCAL_STATIC_LIBRARIES
These are the static libraries that you want to include in your module.
Mostly, we use shared libraries, but there are a couple of places, like
executables in sbin and host executables where we use static libraries instead.
LOCAL_STATIC_LIBRARIES := \
libutils \
libtinyxml
LOCAL_MODULE
LOCAL_MODULE is the name of what's supposed to be generated
from your Android.mk. For exmample, for libkjs, the LOCAL_MODULE
is "libkjs" (the build system adds the appropriate suffix -- .so .dylib .dll).
For app modules, use LOCAL_PACKAGE_NAME instead of
LOCAL_MODULE. We're planning on switching to ant for the apps,
so this might become moot.
LOCAL_MODULE_PATH
Instructs the build system to put the module somewhere other than what's
normal for its type. If you override this, make sure you also set
LOCAL_UNSTRIPPED_PATH if it's an executable or a shared library
so the unstripped binary has somewhere to go. An error will occur if you forget
to.
Instructs the build system to put the unstripped version of the module
somewhere other than what's normal for its type. Usually, you override this
because you overrode LOCAL_MODULE_PATH for an executable or a
shared library. If you overrode LOCAL_MODULE_PATH, but not
LOCAL_UNSTRIPPED_PATH, an error will occur.
These are the static libraries that you want to include in your module without allowing
the linker to remove dead code from them. This is mostly useful if you want to add a static library
to a shared library and have the static library's content exposed from the shared library.
Any flags to pass to invocations of yacc for your module. A known limitation
here is that the flags will be the same for all invocations of YACC for your
module. This can be fixed. If you ever need it to be, just ask.
LOCAL_YACCFLAGS := -p kjsyy
Implementation Details
You should never have to touch anything in the config directory unless
you're adding a new platform, new tools, or adding new features to the
build system. In general, please consult with the build system owner(s)
(android-build-team) before you go
mucking around in here. That said, here are some notes on what's going on
under the hood.
Environment Setup / buildspec.mk Versioning
In order to make easier for people when the build system changes, when
it is necessary to make changes to buildspec.mk or to rerun the environment
setup scripts, they contain a version number in the variable
BUILD_ENV_SEQUENCE_NUMBER. If this variable does not match what the build
system expects, it fails printing an error message explaining what happened.
If you make a change that requires an update, you need to update two places
so this message will be printed.
In config/envsetup.make, increment the
CORRECT_BUILD_ENV_SEQUENCE_NUMBER definition.
In buildspec.mk.default, update the BUILD_ENV_SEQUENCE_DUMBER
definition to match the one in config/envsetup.make
The scripts automatically get the value from the build system, so they will
trigger the warning as well.
Additional makefile variables
You probably shouldn't use these variables. Please consult
android-build-team before using them.
These are mostly there for workarounds for other issues, or things that aren't
completely done right.
LOCAL_ADDITIONAL_DEPENDENCIES
If your module needs to depend on anything else that
isn't actually built in to it, you can add those make targets to
LOCAL_ADDITIONAL_DEPENDENCIES. Usually this is a workaround
for some other dependency that isn't created automatically.
LOCAL_BUILT_MODULE
When a module is built, the module is created in an intermediate
directory then copied to its final location. LOCAL_BUILT_MODULE is
the full path to the intermediate file. See LOCAL_INSTALLED_MODULE
for the path to the final installed location of the module.
LOCAL_HOST
Set by the host_xxx.make includes to tell base_rules.make and the other
includes that we're building for the host. Kenneth did this as part of
openbinder, and I would like to clean it up so the rules, includes and
definitions aren't duplicated for host and target.
LOCAL_INSTALLED_MODULE
The fully qualified path name of the final location of the module.
See LOCAL_BUILT_MODULE for the location of the intermediate file that
the make rules should actually be constructing.
LOCAL_REPLACE_VARS
Used in some stuff remaining from the openbinder for building scripts
with particular values set,
LOCAL_SCRIPTS
Used in some stuff remaining from the openbinder build system that we
might find handy some day.
LOCAL_MODULE_CLASS
Which kind of module this is. This variable is used to construct other
variable names used to locate the modules. See base_rules.make and
envsetup.make.
LOCAL_MODULE_NAME
Set to the leaf name of the LOCAL_BUILT_MODULE. I'm not sure,
but it looks like it's just used in the WHO_AM_I variable to identify
in the pretty printing what's being built.
LOCAL_MODULE_SUFFIX
The suffix that will be appended to LOCAL_MODULE to form
LOCAL_MODULE_NAME. For example, .so, .a, .dylib.
LOCAL_STRIP_MODULE
Calculated in base_rules.make to determine if this module should actually
be stripped or not, based on whether LOCAL_STRIPPABLE_MODULE
is set, and whether the combo is configured to ever strip modules. With
Iliyan's stripping tool, this might change.
LOCAL_STRIPPABLE_MODULE
Set by the include makefiles if that type of module is strippable.
Executables and shared libraries are.
LOCAL_SYSTEM_SHARED_LIBRARIES
Used while building the base libraries: libc, libm, libdl. Usually
it should be set to "none," as it is in $(CLEAR_VARS). When building
these libraries, it's set to the ones they link against. For example,
libc, libstdc++ and libdl don't link against anything, and libm links against
libc. Normally, when the value is none, these libraries are automatically
linked in to executables and libraries, so you don't need to specify them
manually.
The Sun JDK is no longer in Ubuntu's main package repository. In order to download it, you need to add the appropriate repository and indicate to the system which JDK should be used.
dd if=/dev/zero of=ext4.img bs=1MB count=20 // 20mb 짜리 ext4.img라는 파일을 생성
mke2fs -T ext4 ext4.img //ext4 타입으로 ext4.img를 포맷
mkdir test //테스트용 디렉토리 생성(마운트 포인트로 이용)
mount -t ext4 -o loop ext4.img test/ //마운트
마운트가 성공됬으면(mount 커맨드로 확인), 테스트로 파일을 막 써보자. cd test
touch TEMPFILE
mkdir BABO ...
파일이 제대로 생성됐는지 확인해보고, 언마운트를 해보자 cd .. //test 폴더 안에서 언마운트 할라면 device busy라고 불평한다. umount /dev/loop0 //꼭 loop0이라는 보장은 없고 아마 마운트시 available한 loop이 잡히는 걸로 안다.
//이 부분은 losetup으로 체크할 수 있는 걸로 안다. cd test/ ls -al 로 확인하면 텅 빈 걸 확인할 수 있다.
다시 마운트 해보자. mount -t ext4 -o loop ext4.img test/
ls -al 해보면 아까 테스트로 막 만든 파일들이 보인다.
자 그럼 이걸 가지고 뭐에 써먹냐 ? ext4.img 자체를 파티션에 그대로 구울 경우, ext4파일 시스템을 가진 루트파일 시스템 등을 구축 할 수 있는 것이다. EXT4f라고 썼지만 다른 파일 시스템도 이렇게 해서 굽는게 가능하다. 물론 안드로이드용 파일 시스템을 만들 땐 이걸 안 썼다. 언제 저 짓해서 맨날 구워줄 수 있단 말인가? 다른 방법에 대해서는 다음 기회에~