I went along to the second and third days of this event because I've become involved in some start up projects on digital forensics and e-discovery with Brad Glisson. However, the workshop was interesting from an interdisciplinary/socio-techy point of view, as I will try and explain below.

There is a copy of the agenda attached [You do not have permission to access this file].

The attendance was a mixture of disciplines including computer science, software engineering, forensics, law, social science, and criminology; and from a number of backgrounds (particularly professional practitioners and academics. Folks I met included Buckhard Schafer and Richard Jones (Edinburgh Law School), Stephen Mason, Katie Benson (SCDEA), Ian Fergusion, George Weir, Duncan Smeed and Alan Poulter (Strathclyde) as well as some Glasgow folks (Karen, Brad, Mohammed, etc.).

The broad theme of the workshop was the two perspectives (investigation and prevention) on computer based, or computer assisted criminal activity (collectively cyber crime). The cyber security aspect of the workshop considered the challenges of preventing and deterring cyber crime, e.g. legal or market based approaches. The e-investigation parts of the workshop considered the challenges facing e-forensics investigators in terms of both the scale of work and the complexities of modern computer based technologies (cloud computing was discussed here).

The first presentation was by Mike Dickson (Scottish Crime and Drug Enforcement Agency) on the challenges facing e-forensics investigators. Aside from conventional problems of scale (of system storage and complexity) that I've heard Mike talk about before, he raised issues of 'live' investigations of computer systems. Traditional practice for e-investigation is to shut down and image storage from computers siezed during a raid. However, this approach is becoming increasingly problematic with potential evidence being stored on remote servers. Investigators are beginning to develop 'live' investigation techniques - i.e. inspecting a computer system as it is running. However, this raises issues of how the evidence can be presented reliably in court. Related to this are issues of cloud computing - how to access evidence that may be stored on virtualised systems.

Other technical problems raised were the growing use of encryption and forensic wiping tools by `suspects' (I think the word used was actually criminals, but thats LE folks for you. I'm not sure whether Mike was raising these as 'something should be done' or as just facts of life - I can't see a realistic solution to either 'problem'. There seems no point trying to regulate the use of these tools as they have too many perfectly legitimate uses (not least the multi-billion pound e-commerce industry). One thing that does occur to me is that the destruction of evidence is already an offence, so arguably this could be used in some circumstances where records are destroyed.

We discussed challenges in cyber-crime/forensics in groups. Our group raised the issue of accurately measuring the scale of computer based crimes and their cost to society. This seems a natural precursor to any discussion of the need to invest resources or other actions to remedy the problem. This also raised the issue of whether there are better mechanisms than detection and prosecution, based on markets and regulation. The initial challenge seems to be facilitating accurate e-crime reporting from victims.

A second challenge is how to reliably produce evidence from data gathered from computer systems and to store it reliably as well. I got the impression that not many of the participants understood the distinction between data and information. How do we establish a measure of confidence in the evidence produced in court, the same way that we do with other forms of evidence?

Stephen Mason covered the issues of cloud computing and forensics investigations, which can be summed up as "the traditional methods probably won't work". This was a really interesting talk, covering the extent to which existing computer forensics law is out of date. Much of the legislation is geared towards the notion of obtaining goods as evidence pertaining to a crime. Current legislation permits data accessible from a computer to be searched, if it is switched on when discovered during a search. This would seem to raise issues of jurisdiction - where is the evidenceheld, what are the privacy laws there...? Many of the issues relating to cloud computing seem similar to those of corporate governance - how to comply with which set of laws at any one time.

I asked Stephen afterwards how the reliability of electronic evidence is treated in court. I posed the hypothetical situation of a barrister challenging the reliability of each individual piece of evidence based on the completely unknown reliability of the tools that produced it. He suggested judges would be unsympathetic to this approach if all the electronic evidence supported a particular version of events. So, how is the traditional approach to presenting evidence in court compatible with the complexity of evidence produced by a computer? How easily could digital evidence be subverted, either maliciously or by defects in the tools? What is the measure of confidence, and how can it be presented to a jury in a manner that can be evaluated sensibly. Much of the language of the e-forensics people (not Stephen I add) is that if the tools produces some data that can be interpreted as evidence to support your case, then you are done.

I continued the theme of provenance of evidence in the following discussion, including the notion of diversity of tools and results from the dependability field. One point that occurred is that all evidence gathering and presentation is a social process. Evidence is an interpretation of information, which is an interpretation of data. What matters is to be able to convince a jury that your narrative about the evidence is the correct one. In these circumstances, the manner in which the evidence is gathered is bound to come under scrutiny.

George Weir talked in the afternoon about an investigation of risks of identify theft from call centres. This was a good piece of empirical investigation into security practices around pretty sensitive information (which actually shouldn't be - it shouldn't matter if someone knows my mother's maiden name) Summary: be afraid.

Muhammad Nuh Alazar (Nuh) (a Sri Lankan police officer) gave a talk about one particular investigation of ATM card swipers. A few things came out of this. One is that some bank cards apparently still store the PIN on the magnetic strip, including cards from Europe. He also mentioned a privilege escalation vulnerability in the default configuration of certain ATMs, that is openly available in the machine's manual - ask me offline and I'll give you the details.

Nuh also mentioned that his office uses open source tools for some parts of the work. So, this raises the question of what certification process (if any) is used for these tools, and whether this process is necessary for court evidence (apparently no and no, although the FBI do certify some tools such as FTK and Encase, which is why they are so expensive!). There apparently seems to be a reliance on guidance produced by ACPO for producing electronic evidence, without any deeper consideration of the reliability of tools.

At the end of the day we were asked to summarise a research agenda/idea/proposal for cyber crime, cyber security. Much of the proposal was around the idea of educating the general public, although I remain unconvinced as to the value of this. Our group proposed an open source infrastructure for evaluting, comparing and certifying forensic investigation tools.

One of the most interesting on-going discussions of the workshop was between the computer science dependability people (well, mostly me :-) ) and the e-forensics investigators concerning the reliability and provenance of evidence produced from e-forensics investigations. From what I could tell, e-forensics investigators have a back ground in law enforcement, the law or just about anything other than computer science and software engineering. They typically take a masters degree in something like E-Forensics and Digital Discovery at Glasgow which focuses on the legal aspects of evidence gathering and the current IT skills required based on the current commercial tools (FTK,Encase). Perhaps unsurprisingly, it was surprisingly hard to get over to the forensics people that computer systems are incredibly complicated things, whose outputs cannot be trusted at face value, particularly in matters of criminal evidence. Arguing that the people using the tools are experts (when they are manifestly not expert computer scientists) or that they are following established guidelines appears unsatisfactory to me if those guidelines paper over the problems of transparency. We had the same arguments for years in the voting systems field trying to convince vendors and election administrators that computers can easily made to misrepresent their internal state on output.

I'm not saying that electronic evidence should not be used in court, but I think we have a long way to go in establishing electronic evidence as reliable and transparent. My final point to the forensics folks was what happens when a major defect is found in one of the tools? How will you respond? Will all convictions be quashed? Who would ever trust electronic evidence again?

Not socio-technical systems, but of interest to many folks on this site!

http://www.bbc.co.uk/programmes/b00s0vd4

The Digital SYstems KTN abd the Digital Communications KTN are jointly organising a series of workshops on Cloud computing.  The first one is at BT in Martlesham.

They are looking for more venues and speakers.  We are looking to organise one in Bristol.

Contact is Mahesha Pandit - email Mahesha.Pandit@Intellectuk.org

Closing date is 30 April 2010.

according to the advert: "We are particularly interested in candidates working in the areas of: implementation of safety critical, dependable and/or secure distributed systems."

Full details at:

http://www22.i-grasp.com/fe/tpl_glasgow01.asp?newms=jj&id=32056

(the hostname really is a UoG site.).

I thought this might be of interest to LSCITS/INDEED folks. I attended a meeting yesterday in London organised by Ian Nussey of IBM as part of their "Education for a Smarter Planet" initiative (Some material on the Smarter planet initiative can be found at http://www.redbooks.ibm.com/redpapers/pdfs/redp4564.pdf). The purpose of the meeting was to discuss how software engineering will be taught in the next 10-15 years. The IBM position is that the scale of many software projects (in terms of components, data, connections,pervasiveness) will present a challenge to software engineering graduates that they are not currently prepared for, in terms of both construction and maintenance. IBM are currently connecting these things to technologies and concepts such as cloud computing and pervasive computing.

The meeting consisted of about 20 or so academics (including off the top of my head of names I recognised, met or who spoke: Stuart Anderson, John McDermott Nigel Thomas, David De Roure, Anthony Finklestein, Rebecca Eynon and Bill Dutton) 3-4 folks from organisations like the BCS or the e-Skills initiative and about 10-15 IBMers (including Phil Tetlow, Nick Coleman, Chris Nott and Andy Stanford-Clark).

The morning session consisted of 3 presentations by IBMers on industrial perspectives (impact of cloud computing on business processes, data volumes and pervasive computing) and 3 presentations by academics on the challenges faced in teaching large scale engineering methods (David De Roure, Pedro Sampaio, and Bill Dutton:
virtual busineses was mentioned, along with using tools to simulate business processes and an overview of the enterprise IT architecture course at Manchester, Glasgow and elsewhere; and the importance of social acceptance/socio-technical viewpoints on system design and evolution). Quite aside from the engineering challenges, the privacy implications of these new technologies also need to be considered.

Although the meeting was ostensibly about how we go about teaching the appropriate methods to manage these scales of systems, many of the presentations by both IBMers and academics were research oriented - we're trying these techniques, tools etc. Those talks which were more teaching oriented didn't really say anything that was really new "we need to teach analysis and design" - uh huh!

A longer term perspective was suggested that we are scaling up away from precision engineering. This relates to an earlier comment, that we are at the same stage now with system orchestration as we were with programming 40 years ago - lack of structure etc.

A number of folks pointed out that we don't really know how to do this stuff that well ourselves, so trying to decide how we teach the concepts is probably going to be quite hard. It was also quite hard to pin down the IBMers on what skills they thought current graduates lacked. I made the suggestion that we do need to make students aware that they could well come up against these sorts of systems, even if the methods for managing them are not well understood.

A question was raised along the lines of "okay, we can try and teach large scale system engineering", what should we drop to accomodate it, which wasn't really answered very convincingly (no one wants to say subject X isn't needed anymore). The implication is that university course should become more diverse, producing a wider range of graduates. This would seem to be a real challenge in the current funding situation, with computing science departments being asked to teach more with less resources. The suggestion was made that computing departments need to coordinate with other disciplines (e.g. business schools) to develop course that encourage graduates to think about software engineering in terms of business requirements. One example given was team projects containing mixes of business, management and software engineering students. In principle this is fine, but there are often pretty challenging logistical problems in making these sorts of cross departmental projects work, and it certainly doesn't halve the effort of coordinators in individual departments.

Another challenge for universities is the provision of realistic scenarios that let students experience large scale or pervasive engineering challenges. This seems an obvious area where industry can help, and IBM (and presumably others) seem to have done quite a bit setting up labs to support this work. A question for the St Andrews folks: have you considered an undergraduate course on SoS/Cloud computing, perhaps using the cluster you've built?

The afternoon session was split between break out session on security,
engineering and professionalism. I attended the security session for no better reason than that it was poorly attended. I felt this was the most productive session, with some concrete ideas produced for next steps. The main points were:

Encouraging/supporting academics to spend time in industry to experience large scale problems

IBM (and others) to provide realistic scale problems with a security/risk management focus

Universities to review their security curricula with respect to industry certification programmes such as CISP. The intention would be to see what 'competencies' are currently provided by the university programmes.

Universities to investigate the extent to which their programmes support the teaching of regulatory and compliance frameworks - this may be an area that could be more easily supported by industrial expertise

The engineering theme seemed to address new engineering concepts/paradigms needed (again, this seems more like research than teaching). There was a suggestion made that software engineering students should be required to write software with user experience and object design in mind. Quite an odd suggestion that universities don't do this - I would contend that I certainly do. However, its true that we don't explicitly require students to consider the HCI aspects of their design (although we do expect IUs to be de-coupled from the systems). One point made was that the assumption of one system, one user and one interface doesn't hold. This is certainly true, but at least we teach the engineers to decouple the model from the view and control in order to facilitate this situtation.

A couple of ideas were suggested for:

Non-Linear projects - have each software team inherit a code base from another team so that they experience hand-over processes. This goes back to a conversation I had with Q a while ago about the emphasis we place on product rather than process when assessing students.

Try managing softwware projects of distributed teams of students, e.g. across multiple university sites. This is quite a nice idea, although again, the collaboration issues are challenging. Apparently, Newcastle, Leeds and (York?) have tried this with some success. I'm trying to do something similar on a smaller scale over the summer - have my level 3 project team manage themselves as an open source development.

Here is the agenda for the meeting [You do not have permission to access this file].

According to the Register, UK.gov plans to implement a "cheaper, smarter and greener" IT infrastructure, in order to save a cool £3.4billion.

http://www.theregister.co.uk/2010/01/27/government_it_plan/

One of the slightly scary aspects of this is the establishment of an Application Store (presumably the consultants who wrote the report banked on one or civil servants and ministers having an iPhone), which suppliers could use as the basis of a "pay per use" business model, without having to go through normal procurement rules. While this seems superficially attractive (transferring risk to developers from the public sector), these applications will could be used to handle sensitive data. How do they plan to regulate which applications different departments can use? Which suppliers will be permitted to upload applications to the store (do they envisage an Apple-like regulation system)?

A nice story of (a) the trick of getting atomic transactions right and
(b) human recovery of undependable systems.

We (Amanda and myself) were in Glasgow to spend some of the vouchers
we were given back in May (about £240 in total) . We'd finally
managed to choose a box of cutlery for about £150 (only took an hour)
and approached the till at Debenhams to pay for them. We were
informed that the tills had been behaving unreliably with vouchers for
most of the morning, but that the sales assistant would try to make
the transaction for us.

The assistant scanned the cutlery barcode and then attempted to enter
the voucher (several times). At this point the till froze completely.
We waited about 5 minutes until the tills were reset, when the sales
assistant began processing the sale again. He scanned the cutlery
again, scanned the voucher (successfully!) and then asked us for the
remaining balance of £60. A little confused, we explained to him that
the voucher should have a value of over £240 and showed him the
accompanying receipt to indicate this. A supervisor then said she
would check the voucher card's transaction history and departed.

While we were waiting, it suddenly occured to Amanda that the missing
money on the voucher was exactly equal to the cost of the cutlery.
The sales assistant checked the till's transaction history - nothing.
The till had debited the voucher, but not completed the transaction,
and apparently there was no roll-back mechanism (or it had failed) for
returning the money to the voucher. The supervisor returned with the
voucher and transaction history, which also did not show the cutlery.
To correct the problem, the supervisor gave us a hand written receipt.
Presumably, the audit trail would have to record the `missing' cutlery
as being stolen or missing (since its sale could not be entered onto
the database).

I speculate that because the company has already collected the
value for the sale (as a voucher, rather than the cutlery), they
manage voucher sales in a similar way to exchange of goods. As such,
the software is probably less robust than that used for actual
transactions.

Important Dates

Submission deadline	1 November 2009
Author Notification:	1 December 2009
Final Manuscripts Due:	15 January 2010
Registration Due:	15 January 2010
Workshop Date:	20 April 2010

One of the problems with resilience is that we can recognise it after the fact, but it's hard to assess before something goes wrong. Exercises are the common way of testing this but these are (a) expensive and (b) often unrealistic.

Given the notions of resilience (persistence of dependability in the face of change, recovery from failures), I wonder if we could assess the resilience of a system by assessing the heterogeneity of the demands that it successfully handle. The argument being here that if a system/organisation has the capacity to deal with diverse demand, then failure of some kind, is just another type of everyday problem that can be handled.

There is therefore an argument that standard computer systems make a system less resilient because they reduce the heterogeneity that can be handled. The ability to create mashups, etc. make it more resilient because it is therefore better at dealing with heterogeneity.

I realise that this is rambling rather than a coherent hypothesis but is it worth pursuing?

I've been talking to Chen, a visitor from Australia about his work on mashups and this has started a train of thought on how mashups could be used to break the tyranny of process imposed by ERP systems. Essentially, ERP systems are designed to support data aggregation, where all of an enterprise's data is in one place and can be accessed by different applications, which are part of the system. Common rules and processes are defined for the enterprise using this data.

While this seems to be a good idea in principle, in practice, many organisations have terrible problem with ERP system deployment because, for good reasons, different bits of the organisation work in different ways. For example, in a university, some departments are recruiters (they have to actively seek students) and others are selectors (they have too many applicants). An ERP system may try to impose the same process on each but if they are to operate effectively, they really need different processes.

While ERP systems can handle some diversity, the reality is that the organisation changes faster than the system can and there will always be pressures for different ways of working to be supported. So, it seems to me that an integral part of enterprise system design should be to include system features that allow end users to create mashups, while stlll maintaining an enterprise database. SAP and Microsoft have recognised this in their Duet product, which allows 'simple' (with SAP, simple is a relative concept) linking of SAP with Office. IBM, more generally, have a business mashups product, which is designed for opportunistic end-user development. This seems to be able to link to ERP systems.

How does this relate to LSCITS in general? Well, if we take the notion of an LSCITS as a digital ecosystem , then perhaps a central feature of this system should be a capability to create mashups so that heterogeneous ways of working can be supported. It's impossible to anticipate what the needs might be nor to predict when a new need will arise. We can't therefore plan for this and allow effort for 'centralised' development of new features.

It also means that we should be thinking about guidelines for system design where the assumption should be that the data from the individual system components will be used in unanticipated ways. This is not just about exposing an API - it's also means thinking about using RSS feeds to indicate when changes occur and, critically, having fine-grain security, so that confidentiality is only maintained when it's really necessary.

I am wondering now if a resilient system is actually one that can support heterogeneous ways of working, where some components of the system may not always behave in the ways that we anticipate. So, the issue is not recovering from failure but adapting to a new way of working.