Databases
StackOverflow2013
Postsdbo
POBypassing I/O scheduling and linux kernel page buffering
 

Note that there are some explanatory texts on larger screens.

plurals
  1. POBypassing I/O scheduling and linux kernel page buffering
    primarykey
    data
    text
    <p>What I wanna trying to accomplish:</p> <pre><code>Developing an linux application in C language, that "exclusively" accesses a </code></pre> <p>PATA/SATA hard disk drive(HDD) to send ATA commands(in fact only those ATA commands that do not modify any byte on HDD accessed - eg. READ_SECTOR, IDENTIFY_DEVICE, SET_FEATURES, etc.).</p> <p>By "exclusively", I mean that as soon as HDD is powered on(a custom hardware-which is a simple on-off switch, ensures that HDD is not powered on till the application is loaded and desires to do so), the first and only access to that HDD is only to my application. IOWs except my application, not even the linux kernel(including SCSI sub-system) nor any other application or process or human user will ever be able to access that HDD, unless when my application instructs/permits them to do so.</p> <p>There is another requirement on my application: As the access to HDD is quite critical(in terms of control and not in terms of performance) in our app., so it is not desired that any I/O schedular is involved in the transactions done by application(performance on this HDD is not a constraint.). Also it is not desired that the data read from HDD is buffered by kernel buffer or page buffer. The application will read in block size of 512 bytes or a multiple of it only. </p> <p>Now the problem that I'm facing is:</p> <p>The SCSI sub-system resides below(and written to work with) I/O scheduler and kernel buffer or page buffer cache.</p> <p>Although 'sg-driver' is provided by the SCSI subsystem to directly send commands(- Linux SCSI sub-system commands, not ATA or SCSI commands directly - which are then translated by libata to actual ATA commands. Am I right here?) to the HDD,but that is a I/O approach - you give i/p and get o/p, i.e. you have no control over the process of data transfer protocol (eg PIO, DMA and ATA status and Error registers, etc.) and device configuration(via Set Features ATA command.).</p> <p>Also the error reporting mechanism must be sound and is specific to ATA protocol and not simply Linux SCSI subsystem error codes. IOWs my application need to have access to ATA error register and ATA status register on PATA/SATA HDDs.</p> <p>What my application demands is exclusive control of HDD - eg. issuing a READ_SECTOR ATA cmd and then retrieving the data itself from HDD directly via reading i/o ports or via 'libata' with the above requirements must be satisfied.</p> <p>What I can't do ?</p> <p>I'm not going to write a PATA/SATA HBA device driver or every HBA available in market, as those are already included in kernel for libata.</p> <p>What I learned till now ?</p> <p>To accomplish the desired task, I may(or may not?) need to write a block device driver that interacts directly with VFS layer(or is there any way to bypass VFS even, so that my app. can directly communicate with this block driver) w/o involving/messing with kernel buffer or page buffer and I/O scheduler. This block driver will communicate directly with libata(bypassing SCSI subsystem upper layer ), which then communicates with PATA/SATA HBA driver.</p> <p>Is it possible to write such a driver in a cpu architecture independent way?</p> <p>Is it a feasible approach? If yes then would it affect I/O performance of other HDDs attached that are not accessed by my app. in this way. Do I need to write a system call over VFS(or bypassing it if possible)in this case for my application to communicate wih my block driver? Please enlighten me regarding this approach.</p> <p>or is it possible for my block device driver to directly communicate with PATA/SATA HBA driver written for libata, But again would this approach affect I/O performance of other HDDs attached that are not accessed by my app. in this way. Also how would my application communicate with this block device driver? </p> <p>Please enlighten me.</p> <p>Also I wanna know about the same scenario for my application, with one difference - instead of PATA/SATA drives what if I have SCSI hard drives and its variants - specifically, SAS, Fibre channel and USB. And of course this time i'll not be using libata and ATA commands, but SCSI protocol commands.</p> <p>Would u like to suggest a live cd distro as my application host, that contains PATA/SATA HBA libata drivers(- not for IDE sub-system as I will not be using it for it is depreciated now and hence might not be updated with regards for HBA drivers.) for most HBAs.</p> <p>In short, what is the most direct way for an linux application to access a PATA/SATA or SCSI/SAS/Fibre Channel HDD. </p> <p>I hope, I have provided sufficient info regarding my question, but if u wanna get more info or more clarification, please feel free to ask.</p> <p><em><strong>Update1 (on 27/6/2012)</strong>:</em><br> With the useful discussion with Chris(see below) and my research, I reached the following conclusions:</p> <ol> <li><p>that a readymade USB-to-PATA/SATA adapter will not solve my purpose, becoz it does not allow my app. or driver to change the data transfer mode(PIO vs DMA) on-the-fly, it does not allow my app. or driver to read ATA registers.</p></li> <li><p>that a custom made USB-to-PATA/SATA adapter might help, but that will require either a embedded processor which need to implement ATA protocol, or an FPGA chip that implements the whole ATA protocol. But the embedded processor solution involves GPIO and is not good for SATA as it will require specialized transceivers, and the I/O performance will be an issue for both PATA and SATA- too slow for my application. </p></li> </ol> <p>Such a adapter will talk to my linux-kernel driver (or via libusb) to my app. by a custom protocol that helps in communication b/w my app. and ATA protocol on embedded processor. In case of FPGA chip solution, I need to implement this protocol in FPGA itself alongwith the ATA protocol.</p> <p>But at this point it is infeasible in terms of labour, time and money for me to implement FPGA solution and embedded processor solution. So I'm stuck with <em><strong>software only solution</strong></em>.</p> <p>Finally, it seems that I'm probably going to have to duplicate and modify everything down to the hardware inteface layer to meet by requirements as said by Chris.</p> <p>So, <em><strong>between VFS layer and HBA driver or libata layer</strong>, exactly how should I proceed. What things need to be implemented and what not?</em></p> <p>Can someone throw light on this issue? Any ideas??</p> <p><em><strong>Update2 (on 1/7/2012)</strong>:</em><br> I'm struggling with the issue. Is there someone on SO, who can enlighten me? </p>
    singulars
    1. This table or related slice is empty.
    1. This table or related slice is empty.
    1. PTQuestion
    1. USjacks
    1. USjacks
    plurals
    1. This table or related slice is empty.
    1. This table or related slice is empty.
    1. This table or related slice is empty.
    1. PO
      singulars
      1. PTAnswer
    1. VO
      singulars
      1. POBypassing I/O scheduling and linux kernel page buffering
      1. USNikolai Fetissov
      1. VTFavorite
    2. VO
      singulars
      1. POBypassing I/O scheduling and linux kernel page buffering
      1. This table or related slice is empty.
      1. VTUpMod
    1. This table or related slice is empty.
 

Querying!

 
Guidance
A row detail

Detail views are divided into sections. All the information in the data section comes from columns in the selected row. The other sections display data from other, related rows.

Related data can be related in a to-one or a to-many fashion. Captions of data related in a to-many fashion link to a list view showing a filtered view of the table.

Try moving around until you find a non-empty to-many entry and click on the label to get to one. You can move back to the root by clicking on the database name in the header.

SQuiL has stopped working due to an internal error.

If you are curious you may find further information in the browser console, which is accessible through the devtools (F12).

Reload