This chapter describes the process of recording audio and video as well as capturing still images using the Media Engine.
For more information on the Media Session APIs, see the Media Session API reference documentation. To record PCM audio using only the Audio Manager, see Chapter 3, "Audio Subsystem."
The same logic used for playback is used for image and video capture. In both cases a media session must be created to capture and encode the data. Gstreamer plugins are loaded by the source object -- one for encoding and one for rotation.
Recording Audio
The following are the basic steps you should take in a program that records audio using the Media Engine:
- Create a recording session object. When you do, you'll get back a session ID (here, in
g_idsession).AlpMMSessionID gRecSessionID = 0; status = alp_media_session_create( ALP_MM_SESSION_CLASS_RECORD, &gRecSessionID);
- Add a source. Currently, the microphone is the only supported source for audio recording. Set the source type parameter to
ALP_MM_TYPE_AUDIO_RENDER, and specify the URL to the microphone ("dev://mic", orMICDEV).AlpMMSourceID sourceID = 0; status = alp_media_session_add_source(gRecSessionID, ALP_MM_TYPE_AUDIO_RENDER, MICDEV, &sourceID);
Upon return, sourceID will be set to the source ID.
- Next, you need to specify a destination for the recorded audio. To save the audio in a file, the destination URL should be
"dev://filename", orFILENAME.AlpMMDestID destID = 0; status = alp_media_session_add_dest(gRecSessionID, FILENAME, &destID);
- Having specified that you want the recording directed to a file, you next need to specify the path to that file. You do this by setting the destination URL property (
ALP_MM_DESTINATION_URL), like this:int size = 0; size = strlen(filename); status = alp_media_property_set(gRecSessionID, ALP_MM_DESTINATION_URL, ALP_MM_PROPERTY_CODE_STRING, filename, &size);
In the above example,
filenameis a pointer to a C string containing the path to the file, such as "/tmp/record.wav".
IMPORTANT: All of the directories in the specified path must already exist. If you specify a path such as "
/tmp/audio/record.wav" and the /tmp/audio directory does not exist, none of the function calls used to set up and make the recording will return an error. The recording will appear to take place, but the output file won't be created.
- If needed, at this point you can specify other property settings to further customize the recording session. Alternatively you can change the recording properties after the session object has been finalized (see the next step).
For a list of properties that you can set for a recording, see "Media Session Properties - Record and Capture." Note that only some of the properties are relevant for audio recordings.
As an example, you can specify the codec, like this (note that you specify the source ID, rather than the usual session ID when calling
alp_media_property_set()):AlpAudioCodecs codec = ALP_MM_AUDIO_CODEC_RAW; Â // WAV file size = sizeof(codec); status = alp_media_property_set(sourceID, ALP_MM_AUDIO_CODEC_TYPE, ALP_MM_PROPERTY_CODE_INT32, &codec, &size);
Or, you could specify the sample rate, like this:
int rate = 22050; //sample rate size = sizeof(rate); status = alp_media_property_set(gRecSessionID, ALP_MM_FORMATKEY_RAW_AUDIO_SAMPLERATE, ALP_MM_PROPERTY_CODE_INT32, &rate, &size);
- If you wish to be notified when errors and warnings arise, when the session state changes (from playback to stopped, for instance), or on a periodic basis during playback, write an event handler and register it as a media session callback function. See "Registering a Callback" for details.
- "Finalize" the session object, like this:
alp_media_session_finalize(gRecSessionID);
- You start, stop, and pause the recording by issuing "control" commands. To begin the recording:
status = alp_media_session_control(gRecSessionID, Â Â Â Â ALP_MM_SESSION_CTL_STOP);
status = alp_media_session_control(gRecSessionID, Â Â Â ALP_MM_SESSION_CTL_PAUSE);
status = alp_media_session_control(gRecSessionID, Â Â Â ALP_MM_SESSION_CTL_STOP);
If you want to stop the recording and then record another stream to a different file, simply issue a "stop" command, change the destination, and then issue a "run" command, as shown here:
status = alp_media_session_control(gRecSessionID, ALP_MM_SESSION_CTL_STOP); alp_media_property_set(gRecSessionID, ALP_MM_DESTINATION_URL, ALP_MM_PROPERTY_CODE_STRING, (void *)new_filename, (void *)&size); alp_media_session_control(gRecSessionID, Â Â Â ALP_MM_SESSION_CTL_RUN);
- When you are done with the session, dispose of the session and free up its resources:
alp_media_session_remove_all(gRecSessionID); alp_media_session_destroy(gRecSessionID);
Putting it all Together
The code in Listing 5.1 shows one way to write a function that creates a session for the recording of audio through the microphone, the result of which is to be stored in the file indicated by the filename parameter.
Listing 5.1 Audio recording setup
alp_status_t CreateRecordingSession(char *filename){
AlpMMSourceID sourceID;
AlpMMDestID destID;
int size;
alp_status_t status;
AlpAudioCodecs codec = ALP_MM_AUDIO_CODEC_RAW; // WAV file
int rate = 22050; //sample rate
status = alp_media_session_create(
ALP_MM_SESSION_CLASS_RECORD, &gRecSessionID);
if(status == ALP_MM_STATUS_OK){
status = alp_media_session_add_source(gRecSessionID,
ALP_MM_TYPE_AUDIO_RENDER, MICDEV, &sourceID);
if(status == ALP_MM_STATUS_OK){
status = alp_media_session_add_dest(gRecSessionID,
FILENAME, &destID);
if(status == ALP_MM_STATUS_OK){
size = sizeof(codec);
status = alp_media_property_set(sourceID,
ALP_MM_AUDIO_CODEC_TYPE,
ALP_MM_PROPERTY_CODE_INT32, &codec, &size);
if(status == ALP_MM_STATUS_OK){
size = sizeof(rate);
status = alp_media_property_set(gRecSessionID,
ALP_MM_FORMATKEY_RAW_AUDIO_SAMPLERATE,
ALP_MM_PROPERTY_CODE_INT32, &rate, &size);
if(status == ALP_MM_STATUS_OK){
size = strlen(filename);
status = alp_media_property_set(gRecSessionID,
ALP_MM_DESTINATION_URL,
ALP_MM_PROPERTY_CODE_STRING, filename,
&size);
if(status == ALP_MM_STATUS_OK){
status = alp_media_session_finalize(
gRecSessionID);
}
}
}
}
}
}
return status;
}
Once the session has been created, starting and stopping the recording is simply a matter of sending the appropriate control commands to the session. The code in Listing 5.2 is designed to be called when the user presses a specific GTK+ button (initially labeled "Rec"); it alternately starts and stops the recording, updating the button's label as appropriate.
Listing 5.2 Starting and stopping the recording
void RecordButtonClicked(GtkWidget *button, gpointer data){
alp_status_t status = ALP_MM_STATUS_OK;
if(strcmp(gtk_button_get_label(GTK_BUTTON(button)),
"Rec")){
// stop the recording
status = alp_media_session_control(gRecSessionID,
ALP_MM_SESSION_CTL_STOP);
gtk_button_set_label(GTK_BUTTON(button), "Rec");
} else {
// begin the recording
status = alp_media_session_control(gRecSessionID,
ALP_MM_SESSION_CTL_RUN);
gtk_button_set_label(GTK_BUTTON(button), "Stop");
}
}
When done with the session, be sure to clean up as described in step 9 on page 68.
Recording Video
When recording video there is usually both a video stream as well as an audio stream. As well, two destinations are usually required: one to render the video to the screen (allowing the user to preview what they are recording), and one to write the file that contains both audio and video
The sequence of steps you use when recording video is much the same as when recording audio. Here are the basic steps:
- Create a recording session object. When you do, you'll get back a session ID (here, in
g_idsession).AlpMMSessionID g_idsession = 0; alp_media_session_create( ALP_MM_SESSION_CLASS_RECORD, &g_idsession);
- Add a source. Since this is a video recording, specify the camera device. (The recording of a video telephony session is currently not supported). Set the source type parameter to
ALP_MM_TYPE_VIDEO_RENDER, and specify the URL to the camera ("dev://camera", orCAMERADEV).AlpMMSourceID g_idcamerasource = 0; alp_media_session_add_source (g_idsession, ALP_MM_TYPE_VIDEO_RENDER, CAMERADEV, &g_idcamerasource);
Upon return, g_idcamerasource will be set to the source ID.
- If you want to monitor the recording as you are making it, you need to specify the LCD as a destination device. You can also specify the portion of the display to be used to monitor the recording by specifying a destination rectangle property.
This step is optional and can be skipped if you don't need to monitor the recording.
AlpMMDestID g_idlcddest; alp_status_t status; AlpMMRectangle rect; int32_t size = sizeof(AlpMMRectangle); status = alp_media_session_add_dest (g_idsession, LCDDEV, &g_idlcddest); if(status == ALP_MM_STATUS_OK){ // Specify the coordinates and dimensions of the rectangle // to be used when monitoring the recording session. rect.x = 0; rect.y = 32; rect.width = 240; rect.height = 240; status = alp_media_property_set(g_idlcddest, ALP_MM_SESSION_DEST_RECT, ALP_MM_PROPERTY_CODE_RAW, (void *)&rect, &size); }
- Next, you need to specify a destination for the recorded audio. To save the audio in a file, the destination URL should be
"dev://filename", orFILENAME.alpMMDestID g_idfiledest = 0; alp_media_session_add_dest (g_idsession, FILENAME, &g_idfiledest);
- Having specified that you want the recording directed to a file, you next need to specify the path to that file. You do this by setting the destination URL property (
ALP_MM_DESTINATION_URL), like this:char filename[128]; int32_t size = sizeof(filename)/sizeof(char); strcpy (filename, "/tmp/media/wav.wav"); alp_media_property_set(g_idsession, ALP_MM_DESTINATION_URL, ALP_MM_PROPERTY_CODE_STRING, (void *)filename, (void *)&size);
- If needed, at this point you can specify other property settings to further customize the recording session. Alternatively you can change the recording properties after the session object has been finalized (see the next step).
For a list of properties that you can set for a recording, see "Media Session Properties - Record and Capture." Note that only some of the properties are relevant for video recordings.
- Before you can record, you must "finalize" the session object, like this:
alp_media_session_finalize(g_idsession);
- Initiating the video recording session is slightly different than initiating an audio recording session. As well, how you start the recording differs depending upon whether or not you chose to monitor the recording on the LCD in step 3 on page 71.
If you added the LCD as a destination and thus want to monitor the recording as it is being made, begin the recording like this:
alp_media_session_control(g_idsession, ALP_MM_SESSION_CTL_RUN); alp_media_session_control(g_idsession, ALP_MM_SESSION_CTL_RECORD);
If you are not going to monitor the recording, begin the recording this way:
status = alp_media_session_control(g_idsession, ALP_MM_SESSION_CTL_RECORD); status = alp_media_session_control(g_idsession, ALP_MM_SESSION_CTL_RUN);
Why the difference?
ALP_MM_SESSION_CTL_RECORDopens a pipeline between the source (the camera, in this case) and the destination file, but it does not start GStreamer running.ALP_MM_SESSION_CTL_RUNstarts GStreamer. When recording with a preview, issuing the "run" command first starts up GStreamer and sets up the preview on the LCD screen. Then, when you issue the "record" command, the pipeline from the camera to the file is opened and the recording session begins. If you aren't previewing the recording session, you issue the "record" command first to open the pipeline. Then you issue the "run" command to begin the recording session. - You pause and stop a video recording session just as you would when recording audio. To pause:
alp_media_session_control(g_idsession, Â Â Â ALP_MM_SESSION_CTL_PAUSE);
alp_media_session_control(g_idsession, Â Â Â ALP_MM_SESSION_CTL_STOP);
- When you are done with the session, call
alp_media_session_destroy()to dispose of the session and free up its resources.
Recording Video and Audio
To record both audio and video, follow the same sequence of steps as when recording video, but add two source devices—the microphone and the camera—rather than just one. That is, instead of what is shown in step 2 on page 71, do this:
AlpMMSourceID g_idcamerasource = 0; AlpMMSourceID g_idmicsource = 0; alp_media_session_add_source (g_idsession, ALP_MM_TYPE_VIDEO_RENDER, CAMERADEV, &g_idcamerasource); alp_media_session_add_source (g_idsession, ALP_MM_TYPE_AUDIO_RENDER, MICDEV, &g_idmicsource);
Capturing Still Images
Capturing a Picture
With the ACCESS Linux Platform's Media Engine, capturing still images is a simple matter of creating a media session, setting the input source to the camera, and specifying a callback function to handle the captured image. You can specify a portion of the LCD to act as a "viewfinder", allowing the user to preview the image before it is captured by adding the LCD as a destination, much as you do when recording video.
Inside your callback function you receive a raw video data buffer captured as a video frame; you need to compress the raw video data into a specific picture file. Note that the raw video data buffer usually has the format of YUV422 planar; your application can encode into JPEG or PNG.
The following steps walk through the process of writing a program to capture still images from the camera device.
- Create a recording session object. When you do, you'll get back a session ID (here, in
g_idsession).AlpMMSessionID g_idsession = 0; alp_media_session_create( ALP_MM_SESSION_CLASS_RECORD, &g_idsession);
- Add a source, specifying the camera device. Set the source type parameter to
ALP_MM_TYPE_VIDEO_RENDER, and specify the URL to the camera ("dev://camera", orCAMERADEV).AlpMMSourceID g_idcamerasource = 0; alp_media_session_add_source (g_idsession, ALP_MM_TYPE_VIDEO_RENDER, CAMERADEV, &g_idcamerasource);
Upon return, g_idmicsource will be set to the source ID.
- If you want to see the image on the LCD before capturing it, specify the LCD as a destination device. The set the destination rectangle property to specify what part of the display is to be used to preview the image.
This step is optional and can be skipped if you don't need to preview the image.
AlpMMDestID g_idlcddest = 0; alp_status_t status = 0; AlpMMRectangle rect; int32_t size = sizeof(AlpMMRectangle); status = alp_media_session_add_dest (g_idsession, LCDDEV, &g_idlcddest); // Specify the coordinates and dimensions of the rectangle to // be used when monitoring the recording session. rect.x = 0; rect.y = 32; rect.width = 240; rect.height = 240; status = alp_media_property_set(g_idlcddest, ALP_MM_SESSION_DEST_RECT, ALP_MM_PROPERTY_CODE_RAW, (void *)&rect, &size);
- Register a "grab" callback function that will process the captured the data.
status = alp_media_session_register_grab_callback(g_idsession, Â Â Â (AlpMMSessionGrabCallbackFn)Grab, Â Â Â NULL);
Note that you'll need to write this callback function yourself. When the image is captured, it is stored in a buffer and handed off to your callback function along with other data about the image. See the description of the
AlpMMSessionGrabEventdata structure to see what is passed in to your callback.
Listing 5.3 Image capture callback function
void Grab(AlpMMSessionGrabEvent *grab, void *userdata){
int32_t size = grab->buffersize;
int32_t width = grab->width;
int32_t height = grab->height;
g_print ("Image captured. width: %d, height: %d",
width, height);
prv_capture_yuv2jpeg(grab->buffer, grab->size, width,
height, "/tmp/media/jpg.jpg");
}
- If needed, at this point you can specify various camera settings by adjusting property values. Alternatively you can change the property values after the session object has been finalized (see the next step).
For a list of properties that you can set, see "Media Session Properties - Record and Capture." Note that only some of the properties are relevant for still image captures.
- "Finalize" the session object:
alp_media_session_finalize(g_idsession);
- If you designated the LCD as a destination, thus enabling the user to preview their image before capturing it, open a pipeline between the camera and the LCD by issuing a "run" command:
alp_media_session_control(g_idsession, Â Â Â ALP_MM_SESSION_CTL_RUN);
- Whether or not you are using the LCD to preview the image, trigger an image capture by issuing a "grab" command, like this:
alp_media_session_control(g_idsession, Â Â Â ALP_MM_SESSION_CTL_GRAB);
This will invoke your callback function with the pixels captured from the camera.
- When you are done with the session, call
alp_media_session_destroy()to dispose of the session and free up its resources.
See Listing 5.4 for a command-line program that performs an image capture.
#include <string.h> #include <stdio.h> #include <stdlib.h> #include <alp/media_session.h> AlpMMSessionID g_idsession = 0; AlpMMSourceID g_idsource = 0; AlpMMDestID g_iddest = 0; AlpMMDestID g_idfiledest = 0; AlpMMSessionEvent gevent = {0,0, 0, 0, 0, 0, 0}; int size = 0; AlpMMRectangle rect; AlpCameraWhiteBalanceMode whitebalance = ALP_MM_WHITE_BALANCE_AUTO; void capture (AlpMMSessionGrabEvent * grab, void * userdata) { int size = grab->buffersize; int width = grab->width; int height = grab->height; printf ("----------capture----------- width: height: size: %d, %d, %d",width, height, size); // prv_capture_yuv2jpeg(grab->buffer, size, width, height, "/tmp/media/ // jpg.jpg"); } int main (int argc, char *argv[]) { char c; int i = 0; rect.x = 0; rect.y = 32; rect.width = 240; rect.height = 266; alp_status_t status; AlpMMPropertyCode propertyCode = ALP_MM_PROPERTY_CODE_STRING; AlpMMPropertyCode propertyInt = ALP_MM_PROPERTY_CODE_INT32; AlpMMPropertyType propertyType = ALP_MM_PROPERTY_TYPE_DEFAULT; status = alp_media_session_create (ALP_MM_SESSION_CLASS_RECORD, &g_idsession)); status = alp_media_session_add_source (g_idsession,ALP_MM_TYPE_VIDEO_RENDER, CAMERADEV, &g_idsource)); status = alp_media_session_add_dest (g_idsession, LCDDEV, &g_iddest)); status = alp_media_property_set (g_iddest, ALP_MM_SESSION_DEST_RECT, propertyCode, (void*)&rect,(int32_t*)&size)); status = alp_media_session_register_grab_callback(g_idsession, (AlpMMSessionGrabCallbackFn)capture, (void *)1)); status = alp_media_session_finalize (g_idsession)); printf ("Input 'r' begin to run\n"); printf ("Input 'q' quit\n"); while (1) { c = getchar(); switch (c) { case 'r': // run status = alp_media_session_control(g_idsession, ALP_MM_SESSION_CTL_RUN)); printf ("Input 'p' pause record\n"); printf ("Input 's' stop record\n"); printf ("Input 'q' quit\n"); break; case 'p': // pause status = alp_media_session_control(g_idsession, ALP_MM_SESSION_CTL_PAUSE)); printf ("Input 'r' begin to run\n"); printf ("Input 'q' quit\n"); break; case 's': // stop status = alp_media_session_control(g_idsession, ALP_MM_SESSION_CTL_STOP)); printf ("Input 'r' begin to run\n"); printf ("Input 'q' quit\n"); break; case 'c': // capture status = alp_media_session_control(g_idsession, ALP_MM_SESSION_CTL_GRAB)); break; case 'q': // quit status = alp_media_session_remove_all (g_idsession)); status = alp_media_session_destroy (g_idsession)); return 0; break; default: break; } } return 0; }
Media Session Properties - Record and Capture
As part of a media recording or capture session you can set the properties listed in the following sections. Note that currently you cannot use alp_media_property_get() to obtain the value of these properties for a recording session.
IMPORTANT: Always check the status value returned from the
alp_media_property_get() or alp_media_property_set() call; if it is not ALP_MM_STATUS_OK, the operation did not succeed.
ALP_MM_AUDIO_CODEC_TYPE
Purpose
The codec used to encode a recorded audio stream.
Get/Set
Data Type
AlpAudioCodecs (ALP_MM_PROPERTY_CODE_INT32)
Example
AlpAudioCodecs codec_type; codec_type = ALP_MM_AUDIO_CODEC_AMR_WB; status = alp_media_property_set(sessionID, ALP_MM_AUDIO_CODEC_TYPE, ALP_MM_PROPERTY_CODE_INT32, &codec_type, NULL); /* If successful, the specified codec will be used to encode    subsequent audio streams */
ALP_MM_DESTINATION_ROTATE
Purpose
Specifies the video image rotation mode. Use this to rotate the preview image on the device's LCD.
Get/Set
Data Type
AlpMMDestinationRotateType (ALP_MM_PROPERTY_CODE_INT32)
Example
AlpMMDestinationRotateType rotate_setting; rotate_setting = ALP_MM_DESTINATION_ROTATE_90L; status = alp_media_property_set(sessionID, ALP_MM_DESTINATION_ROTATE, ALP_MM_PROPERTY_CODE_INT32, &rotate_setting, NULL); /* If successful, the preview image displayed on the LCD Â Â Â will be rotated 90 degrees to the left. */
ALP_MM_DESTINATION_URL
Purpose
The URL used to create a destination object.
Get/Set
Data Type
char * (ALP_MM_PROPERTY_CODE_STRING)
Comments
The alp_media_property_set() function makes a copy of the supplied URL string. The supplied URL must not exceed 256 characters (including the terminating NULL character). The length parameter must be a valid pointer, but the value to which it points is ignored.
Example
gchar filename[128]; int32_t length = 0; strcpy (filename, "/tmp/media/wav.wav"); status = alp_media_property_set(sessionID, ALP_MM_DESTINATION_URL, ALP_MM_PROPERTY_CODE_STRING, filename, &length); /* If successful, the recording will be stored in the file    indicated by filename. */
ALP_MM_FORMATKEY_RAW_AUDIO_BITS
Purpose
The actual number of sample bits.
Get/Set
Data Type
unit32_t (ALP_MM_PROPERTY_CODE_INT32)
Comments
Attempts to change this property are currently ignored. At this time only 16-bit samples are supported.
ALP_MM_FORMATKEY_RAW_AUDIO_
SAMPLERATE
Purpose
Get/Set
Data Type
unit32_t (ALP_MM_PROPERTY_CODE_INT32)
Comments
Attempts to change this property are currently ignored. At this time the only supported sample rate is 44.1kHz.
ALP_MM_SESSION_DEST_RECT
Purpose
The destination rectangle to use for a video or still image preview. The rectangle specifies both the position and the size.
Get/Set
Data Type
AlpMMRectangle (ALP_MM_PROPERTY_CODE_RAW)
Example
AlpMMRectangle rect; int length = sizeof(rect); rect.x = 0; rect.y = 32; rect.width = 240; rect.height = 266; status = alp_media_property_set(sessionID, ALP_MM_SESSION_DEST_RECT, ALP_MM_PROPERTY_CODE_RAW, &rect, &length); /* If successful, once the "run" command is issued the    preview image will be displayed in the specified portion    of the LCD. */
ALP_MM_SOURCE_CAMERA_BRIGHTNESS
Purpose
A value indicating the camera's "brightness."
Get/Set
Data Type
unit32_t (ALP_MM_PROPERTY_CODE_INT32)
Comments
Brightness values range from -2 to +2. The actual effect they have on the camera depends upon the camera driver and plugins.
Example
uint32_t brightness; brightness = 1; status = alp_media_property_set(sessionID, ALP_MM_SOURCE_CAMERA_BRIGHTNESS, ALP_MM_PROPERTY_CODE_INT32, &brightness, NULL); /* If successful, once the "run" command is issued the    preview image will be displayed in the specified portion    of the LCD. */
ALP_MM_SOURCE_CAMERA_COLOR_EFFECT
Purpose
The color effect to be applied to the image: sepia, black-and-white, etc.
Get/Set
Data Type
AlpCameraColorEffect (ALP_MM_PROPERTY_CODE_INT32)
Example
AlpCameraColorEffect color_effect = ALP_MM_COLOR_EFFECT_BLACK_AND_WHITE; status = alp_media_property_set(sessionID, ALP_MM_SOURCE_CAMERA_COLOR_EFFECT, ALP_MM_PROPERTY_CODE_INT32, &color_effect, NULL); /* If successful, subsequent images (video or still) will be    in black and white. */
ALP_MM_SOURCE_CAMERA_WHITE_
BALANCER
Purpose
Specifies white balance values for a digital camera. White balance adjusts the color synchronization for various lighting conditions.
Get/Set
Data Type
AlpCameraWhiteBalanceMode (ALP_MM_PROPERTY_CODE_INT32)
Example
AlpCameraWhiteBalanceMode white_balance = ALP_MM_WHITE_BALANCE_FLUORESCENT; status = alp_media_property_set(sessionID, ALP_MM_SOURCE_CAMERA_WHITE_BALANCER, ALP_MM_PROPERTY_CODE_INT32, &white_balance, NULL); /* If successful, subsequent images (video or still) will be    adjusted to compensate for fluorescent lighting. */










