gpt4 book ai didi

c# - 数据集级联和内存消耗

转载 作者:IT王子 更新时间:2023-10-28 23:38:29 24 4
gpt4 key购买 nike

我有一个应用程序,它使用 DataSet.WriteXML 导出数据并使用 DataSet.ReadXML 导入数据。在导入过程中,我需要更改某些主键作为应用程序逻辑的一部分。

当记录超过 500K 时,它会成功写入 XML 并从 XML 中读取。一旦我更改了主键,它会等待一段时间并引发 OutOfMemory 异常。我相信的原因是,它必须进行大量级联更新。我在主键更改期间尝试了 BeginEdit 和 EndEdit,但在这种情况下仍然在 EndEdit 中失败。

据我了解,DataSets 还将一些以前的数据保存在内存中。有什么方法可以优化 DataSet 更新操作以消耗最少的内存?

最佳答案

如果您需要更多控制权,则需要删除数据集为您提供的一些功能。减少由级联引起的内存的一种方法是简单的不要级联。使用表模式手动更新表 ID。

这个想法是您可以控制更新哪些行、随时接受更改、强制 GC 中间更新或您可能想要控制的任何其他内容。

我创建了一个简单的测试场景来说明我的意思:

enter image description here

架构:

<?xml version="1.0"?>
<xs:schema id="NewDataSet" xmlns="" xmlns:xs="http://www.w3.org/2001/XMLSchema" xmlns:msdata="urn:schemas-microsoft-com:xml-msdata">
<xs:element name="NewDataSet" msdata:IsDataSet="true" msdata:UseCurrentLocale="true">
<xs:complexType>
<xs:choice minOccurs="0" maxOccurs="unbounded">
<xs:element name="Planet">
<xs:complexType>
<xs:sequence>
<xs:element name="ID" type="xs:int" />
<xs:element name="Name" type="xs:string" minOccurs="0" />
</xs:sequence>
</xs:complexType>
</xs:element>
<xs:element name="Continent">
<xs:complexType>
<xs:sequence>
<xs:element name="ID" type="xs:int" />
<xs:element name="PlanetID" type="xs:int" />
<xs:element name="Name" type="xs:string" minOccurs="0" />
</xs:sequence>
</xs:complexType>
</xs:element>
<xs:element name="Country">
<xs:complexType>
<xs:sequence>
<xs:element name="ID" type="xs:int" />
<xs:element name="ContinentID" type="xs:int" />
<xs:element name="Name" type="xs:string" minOccurs="0" />
</xs:sequence>
</xs:complexType>
</xs:element>
<xs:element name="County">
<xs:complexType>
<xs:sequence>
<xs:element name="ID" type="xs:int" />
<xs:element name="CountryID" type="xs:int" />
<xs:element name="Name" type="xs:string" minOccurs="0" />
</xs:sequence>
</xs:complexType>
</xs:element>
<xs:element name="City">
<xs:complexType>
<xs:sequence>
<xs:element name="ID" type="xs:int" />
<xs:element name="CountyID" type="xs:int" />
<xs:element name="Name" type="xs:string" minOccurs="0" />
</xs:sequence>
</xs:complexType>
</xs:element>
<xs:element name="Street">
<xs:complexType>
<xs:sequence>
<xs:element name="ID" type="xs:int" />
<xs:element name="CityID" type="xs:int" minOccurs="0" />
<xs:element name="Name" type="xs:string" minOccurs="0" />
</xs:sequence>
</xs:complexType>
</xs:element>
<xs:element name="People">
<xs:complexType>
<xs:sequence>
<xs:element name="ID" type="xs:int" />
<xs:element name="StreetID" type="xs:int" />
<xs:element name="Name" type="xs:string" minOccurs="0" />
</xs:sequence>
</xs:complexType>
</xs:element>
<xs:element name="Job">
<xs:complexType>
<xs:sequence>
<xs:element name="ID" type="xs:int" />
<xs:element name="PeopleID" type="xs:int" />
<xs:element name="Name" type="xs:string" minOccurs="0" />
</xs:sequence>
</xs:complexType>
</xs:element>
<xs:element name="Pets">
<xs:complexType>
<xs:sequence>
<xs:element name="ID" type="xs:int" />
<xs:element name="PeopleID" type="xs:int" minOccurs="0" />
<xs:element name="Name" type="xs:string" minOccurs="0" />
</xs:sequence>
</xs:complexType>
</xs:element>
</xs:choice>
</xs:complexType>
<xs:unique name="Constraint1">
<xs:selector xpath=".//Planet" />
<xs:field xpath="ID" />
</xs:unique>
<xs:unique name="Continent_Constraint1" msdata:ConstraintName="Constraint1">
<xs:selector xpath=".//Continent" />
<xs:field xpath="ID" />
</xs:unique>
<xs:unique name="Country_Constraint1" msdata:ConstraintName="Constraint1">
<xs:selector xpath=".//Country" />
<xs:field xpath="ID" />
</xs:unique>
<xs:unique name="County_Constraint1" msdata:ConstraintName="Constraint1">
<xs:selector xpath=".//County" />
<xs:field xpath="ID" />
</xs:unique>
<xs:unique name="City_Constraint1" msdata:ConstraintName="Constraint1">
<xs:selector xpath=".//City" />
<xs:field xpath="ID" />
</xs:unique>
<xs:unique name="Street_Constraint1" msdata:ConstraintName="Constraint1">
<xs:selector xpath=".//Street" />
<xs:field xpath="ID" />
</xs:unique>
<xs:unique name="People_Constraint1" msdata:ConstraintName="Constraint1">
<xs:selector xpath=".//People" />
<xs:field xpath="ID" />
</xs:unique>
<xs:unique name="Job_Constraint1" msdata:ConstraintName="Constraint1">
<xs:selector xpath=".//Job" />
<xs:field xpath="ID" />
</xs:unique>
<xs:unique name="Pets_Constraint1" msdata:ConstraintName="Constraint1">
<xs:selector xpath=".//Pets" />
<xs:field xpath="ID" />
</xs:unique>
<xs:keyref name="Relation8" refer="People_Constraint1">
<xs:selector xpath=".//Pets" />
<xs:field xpath="PeopleID" />
</xs:keyref>
<xs:keyref name="Relation7" refer="People_Constraint1">
<xs:selector xpath=".//Job" />
<xs:field xpath="PeopleID" />
</xs:keyref>
<xs:keyref name="Relation6" refer="Street_Constraint1">
<xs:selector xpath=".//People" />
<xs:field xpath="StreetID" />
</xs:keyref>
<xs:keyref name="Relation5" refer="City_Constraint1">
<xs:selector xpath=".//Street" />
<xs:field xpath="CityID" />
</xs:keyref>
<xs:keyref name="Relation4" refer="County_Constraint1">
<xs:selector xpath=".//City" />
<xs:field xpath="CountyID" />
</xs:keyref>
<xs:keyref name="Relation3" refer="Country_Constraint1">
<xs:selector xpath=".//County" />
<xs:field xpath="CountryID" />
</xs:keyref>
<xs:keyref name="Relation2" refer="Continent_Constraint1">
<xs:selector xpath=".//Country" />
<xs:field xpath="ContinentID" />
</xs:keyref>
<xs:keyref name="Relation1" refer="Constraint1">
<xs:selector xpath=".//Continent" />
<xs:field xpath="PlanetID" />
</xs:keyref>
</xs:element>
</xs:schema>

还有一些生成测试用例的代码

    private void CreateRows(Int32 MaxBaseRows, Int32 MaxChildRows)
{
dataSet1.Clear();
Int32 RowCount = 0;
Random R = new Random();
foreach (DataTable DT in dataSet1.Tables)
{
Int32 NewCount = R.Next(1, MaxBaseRows);
foreach (var FK in DT.Constraints.OfType<ForeignKeyConstraint>())
{
NewCount = NewCount * R.Next(1, MaxChildRows);
}
for (int i = 0; i < NewCount; i++)
{
DataRow DR = DT.NewRow();
foreach (DataColumn DC in DT.Columns)
{
if (DC.ColumnName == "ID")
{
DR[DC] = DT.Rows.Count;
}
else if (DC.DataType == typeof(Int32))
{
Boolean ValueSet = false;
foreach (var FK in DT.Constraints.OfType<ForeignKeyConstraint>())
{
if (FK.Columns.Contains(DC))
{
DR[DC] = R.Next(0, FK.RelatedTable.Rows.Count);
ValueSet = true;
}
}
if (!ValueSet)
{
DR[DC] = R.Next(0, 10000);
}
}
else if (DC.DataType == typeof(String))
{
DR[DC] = String.Format("{0}{1}", DT.TableName, DT.Rows.Count);
}
}
DT.Rows.Add(DR);
RowCount++;
}
}
label19.Text = RowCount.ToString();
dataSet1.AcceptChanges();
}


private void UpdateUsingCascade()
{
EnableRelations();
GC.Collect();
long Mem = System.GC.GetTotalMemory(false);
if (dataSet1.Tables["Planet"].Rows.Count > 0)
{
dataSet1.Tables["Planet"].Rows[0]["ID"] = new Random().Next(BaseRowCount, BaseRowCount + 10);
}
Mem = System.GC.GetTotalMemory(false) - Mem;
DataSet ds = dataSet1.GetChanges();
Int32 Changes = ds.Tables.OfType<DataTable>().Sum(DT => DT.Rows.Count);
label19.Text = Changes.ToString();
label21.Text = Mem.ToString();
dataSet1.AcceptChanges();
}

private void UpdateManually()
{
DisableRelations();
GC.Collect();
long Mem = System.GC.GetTotalMemory(false);

DataTable DT = dataSet1.Tables["Planet"];
Int32 ChangeCount = 0;
if (DT.Rows.Count > 0)
{
DataColumn DC = DT.Columns["ID"];
Int32 oldValue = Convert.ToInt32(DT.Rows[0][DC]);
DT.Rows[0][DC] = new Random().Next(BaseRowCount + 20,BaseRowCount + 30);
Int32 newValue = Convert.ToInt32(DT.Rows[0][DC]);
foreach (DataRelation Relation in DT.ChildRelations)
{
if (Relation.ParentColumns.Contains(DC))
{
foreach (DataColumn CC in Relation.ChildColumns)
{
foreach (DataRow DR in Relation.ChildTable.Rows)
{
if (Convert.ToInt32(DR[CC]) == oldValue)
{
DR[CC] = newValue;
ChangeCount++;
dataSet1.AcceptChanges();
GC.Collect();
}
}
}
}
}
}
Mem = System.GC.GetTotalMemory(false) - Mem;
label20.Text = ChangeCount.ToString();
label22.Text = Mem.ToString();
dataSet1.AcceptChanges();
}

private void EnableRelations()
{
dataSet1.EnforceConstraints = true;
foreach (DataRelation Relation in dataSet1.Relations)
{
Relation.ChildKeyConstraint.UpdateRule = Rule.Cascade;
}
}

private void DisableRelations()
{
dataSet1.EnforceConstraints = false;
foreach (DataRelation Relation in dataSet1.Relations)
{
Relation.ChildKeyConstraint.UpdateRule = Rule.None;
}
}

关于c# - 数据集级联和内存消耗,我们在Stack Overflow上找到一个类似的问题: https://stackoverflow.com/questions/15058436/

24 4 0
Copyright 2021 - 2024 cfsdn All Rights Reserved 蜀ICP备2022000587号
广告合作:1813099741@qq.com 6ren.com